-- Logs begin at Tue 2022-08-23 21:18:52 UTC, end at Tue 2022-08-23 23:25:29 UTC. -- Aug 23 22:08:30.659859 dv-cinder-842542-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 23 22:08:30.668500 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 23 22:08:30 2022] *** Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: nodename: dv-cinder-842542-4-iscsi Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: machine: x86_64 Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: clock source: unix Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: pcre jit disabled Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: detected number of CPU cores: 4 Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: current working directory: / Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: detected binary path: /usr/bin/uwsgi-core Aug 23 22:08:30.672639 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: your processes number limit is 47680 Aug 23 22:08:30.673752 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: your memory page size is 4096 bytes Aug 23 22:08:30.673752 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: detected max file descriptor number: 2048 Aug 23 22:08:30.673752 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: lock engine: pthread robust mutexes Aug 23 22:08:30.673752 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: thunder lock: enabled Aug 23 22:08:30.673752 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 23 22:08:30.673752 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 23 22:08:30.718939 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: Python main interpreter initialized at 0x559d22a489c0 Aug 23 22:08:30.718939 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: python threads support enabled Aug 23 22:08:30.718939 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: your server socket listen backlog is limited to 100 connections Aug 23 22:08:30.718939 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: your mercy for graceful operations on workers is 90 seconds Aug 23 22:08:30.719430 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: mapped 403077 bytes (393 KB) for 2 cores Aug 23 22:08:30.719541 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: *** Operational MODE: preforking *** Aug 23 22:08:30.719891 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: *** uWSGI is running in multiple interpreter mode *** Aug 23 22:08:30.719891 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: spawned uWSGI master process (pid: 81603) Aug 23 22:08:30.720149 dv-cinder-842542-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 23 22:08:30.721818 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: spawned uWSGI worker 1 (pid: 81612, cores: 1) Aug 23 22:08:30.721818 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: spawned uWSGI worker 2 (pid: 81613, cores: 1) Aug 23 22:08:30.721818 dv-cinder-842542-4-iscsi devstack@keystone.service[81603]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 23 22:08:35.588841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:08:35.588841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Aug 23 22:08:35.590144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:08:35.590144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: from cryptography.utils import int_from_bytes Aug 23 22:08:35.639453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:08:35.639453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: from cryptography.utils import int_from_bytes Aug 23 22:08:35.640435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 23 22:08:35.640435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: from cryptography.utils import int_from_bytes Aug 23 22:08:35.643006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 22:08:35.643204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 22:08:35.643407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] command line args: [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 22:08:35.643637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 22:08:35.643834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ================================================================================ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 22:08:35.644071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.644319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_dir = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.644507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.644694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] config_source = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.644876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] debug = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.645098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.645459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.645656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.645876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.646128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.646363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.646551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_config_append = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.646741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.646917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_dir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.647087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.647251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.647538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.647948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.648761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.649050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.649352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.649489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.649672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.649883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.650101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.650334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.650575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.650830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.651084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.651369 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.651877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.652125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] publish_errors = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.652314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.652526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.652748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.652946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.653205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.653449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] standard_threads = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.653682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.653933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.654111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.654289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_journal = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.654469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_json = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.654644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_stderr = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.654820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] use_syslog = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.655056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.655320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.655563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.655829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.656165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.656314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.656554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.656808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.657066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.657325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.657579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.657834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.658072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.658324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.658564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.658817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.659067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.659310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.659592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.659831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.660087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.660328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.660587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.660852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.661131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.661387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.661623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.661860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.662096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.662334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.662587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.662873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.663922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.664794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.665092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.665359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.665608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.665868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.666110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.666353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.666597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.666835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.667071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.667309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.667546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.667787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.668044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.668281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.668525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.668762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.669004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.669267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.669522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.669758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.670010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.670232 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.670480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.670724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.670964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.671200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.671452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.671694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.671938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.672175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.672444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.672684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.672920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.673187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.673428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.673705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.673945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.674197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.674451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.674693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.674930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.675169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.675422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.675769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.675917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.676146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.676392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.676634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.676875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.677140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.686406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.687519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.external = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.password = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.688673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] auth.token = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.689791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.690752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.691704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.692853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.693903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.693903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.693903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.693903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.694588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.694588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.694844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.695500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.695755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.696021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.696290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.696533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.696773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.697084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.697273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.697530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.697785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.698049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.698328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.698582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.698827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.699072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.699307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.699528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.699771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.700011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.700262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.700506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.700760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.701056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.701274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.701528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.701770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.702018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.702272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.702508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.702752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.702985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.703231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.703471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.703881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.704125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.704371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.704628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.704883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.705147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.705385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.705642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.705895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.706136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.706375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.706614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.706848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.707093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.707359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.707921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.707921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.708153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.708328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.708574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.708790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.709056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.709290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.709533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.709766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.710053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.710362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 23 22:08:35.710605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 23 22:08:35.710893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] command line args: [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 23 22:08:35.711027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.711208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.711339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 23 22:08:35.711477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.711607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ================================================================================ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 23 22:08:35.711717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.711931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.712072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] admin_token = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.712199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.712340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_dir = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.712466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.712591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.712719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.712842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] config_source = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.713027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.713237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.713363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] debug = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.713497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.713625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.713793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.713943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.714072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.714202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.714347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.714347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.714573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.714711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.714850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.714977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.715103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.715249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.715373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.715636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_config_append = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.715828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.715954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.716083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.716207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_dir = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.716346 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.716472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.716590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.716706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.716820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.716820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.722008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.722186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.722340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.722472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.722603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.722735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.722880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.723011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.723184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.723324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.723463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.723603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.723731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.723849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.723984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.724149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.724275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.724405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.724530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.724653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.724788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.724925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.725149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.725281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.725281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.726085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.726230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.726396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.726536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.726536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.726536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.726536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.726963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.73/identity {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.727077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.727240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] publish_errors = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.727408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.727523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.727695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.727822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.727948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.728068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.728204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.728314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.728420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.728536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.728669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.728669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.728981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] standard_threads = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.729158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.729326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.729451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.729451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.729724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.729927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.730125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.730348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.730534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_journal = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.730662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.730788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_json = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.731149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.731149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.731391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_stderr = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.731569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.731698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] use_syslog = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.731883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.732025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 23 22:08:35.732153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.732153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.732447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.732615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.732615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.732924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.733974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.734322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.734450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.734450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.734801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.735157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.735508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.735812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.735812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.736124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.736280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.736539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.736774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.736774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.737110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.737277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.737454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.737585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.737585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.737852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.738013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.738154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.738265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.738265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.738550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.738800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.739063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.739321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.739321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.739616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.739858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.740975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.742728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.742925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.742925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.743219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.743346 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.743571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.743766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.743766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.744808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81612) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.745931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.746896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.747834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.748638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.748638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.748905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.749101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.749429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.749749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.750054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.750366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.750666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.751005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.751310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.751612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.751915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.752223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.752563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.752870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.753223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.753544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.753864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.754154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.754445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.754744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.755100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.755424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.755725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.756016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.756308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.756600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.756920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.757236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.757550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.757912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.758119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.758386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.758679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.758963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.759278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.759572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.759844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.760153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.760460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.760746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.761046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.761361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.761636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.761934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.762221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.762529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.762838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.763129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.763408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.external = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.763689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.764035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.764319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.764596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.password = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.764876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] auth.token = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.765194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.765479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.765777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.766057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.766354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.766650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.766948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.767201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.767523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.768184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.768341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.768666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.768967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.769287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.769482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.769649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.769649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.770027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.770027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.770439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.770632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.770921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.771213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.771604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.771889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.772200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.773464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.773464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.773921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.773921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.774299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.774492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.774728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.775039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.775333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.775645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.775963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.776262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.776573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.776947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.777319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.783975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.784992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.785243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.786768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.787515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.787515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.787515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.787515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.787515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.787515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.788030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.788030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.789116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.789527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.789873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.790203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.790545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.790871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.791170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.791483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.791792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.792105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.792409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.792700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.792999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.793343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.793643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.793936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.794387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.794703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.795015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.795329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.795641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.796024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.796307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.796609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.796908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.797305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.797589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.797882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.798175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.798458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.798777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.799050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.799341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.799648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.799949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.800321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.800693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.801100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.801429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.801758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.802090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.802433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.802771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.803110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.803451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.803867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.804195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.804534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.804862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.805235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.805582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.805921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.806245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.806587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.806929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.807279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.807716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.808141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.808507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.808889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.809165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.809468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.809797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.810107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.810418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.810731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.811224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.811519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.811842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.812152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.812461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.812775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.813108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.813431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.813746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.814048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.814343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.814638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.814936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.815249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.815567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.815888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.816186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.816484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.816775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.817104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.817407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.817715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.818041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.818364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.824195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.825905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.826440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.826523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.826651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.826732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.826732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.826732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.826732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.827074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.827316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.827619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 23 22:08:35.827851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81613) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 23 22:08:35.855617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.855870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.856184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.865438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.885110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.885380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.885736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.926883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.939658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.939973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.963670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:35.963903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.112307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.112585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.112725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.112846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.112970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.113196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.113352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.113533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.113664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.113883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.114165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.114446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.139845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.140097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.140284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.140426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.140594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.140734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.140868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.141005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.141179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.141317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.141548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.141866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.142143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.142440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.153635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.153979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.168247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.168588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.168896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.169231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.181148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.181148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.181148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81613) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.182589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.182881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.183170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.183463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.183746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.184023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.184323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81612) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 23 22:08:36.199772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.200088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:08:36.200523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.200675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.200851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:08:36.200998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:08:36.201309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.201457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.201709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.201858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 23 22:08:36.202093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.202234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.202465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.202610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.203116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.203445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.213370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.213795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.214154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.214526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.214676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.215029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.215371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.215763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.222956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.223587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.226634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.227147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.227525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.227894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.228259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.228411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.228648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.228809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.229060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.229207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.229460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.229608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.229850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.230015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.230223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.230365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.230603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.230746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.230985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.231443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.231786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.232125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.254818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.256307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.256596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.257951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.259563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.259912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.261673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.261960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.262247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.264948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.265352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.265698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.268753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.269073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.272542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.273332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.273646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.273866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.274014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.274251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.274615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.274898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.275176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.275456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.278700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.279181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.279688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.280184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.280669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.281081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.281389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.281673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.294121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.294427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.298358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.301658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.301961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.304326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.305165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.306991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.309372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.310167 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.312847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.313226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.315893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.315893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.315893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.315893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.315893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.315893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.317808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.322056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.325432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.325839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.326147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.326509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.326854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.327287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.330788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.331092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.331360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.331626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.338982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.340674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.340954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.343621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.343886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.344177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.344468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.344750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.345456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.345832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.346193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.346536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.355258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.357106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.357292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.357437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.357579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.357746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.357928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.358216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.358577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.358871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.360690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.361095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.361504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.362114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.362273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.376072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.376072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.376072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.376645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.376789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.376929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.377293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.377660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.379028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.382216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.383678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.385163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.385449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.385744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.392279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.393790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.395128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.396776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.397097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.397765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.397765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.399582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.402877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.404396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.405783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.406002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.411080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.411080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.411523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.411523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.413940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81613) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.414453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.414835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.415198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.415409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.415644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.416006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.416380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.416761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.417134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.417441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.417598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.417882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.418224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.418590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.418966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81613) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.421052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.421456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.423870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.426751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.437072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.437478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.437848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.445962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81612) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 23 22:08:36.446360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.446731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.447099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.447474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.447866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.448245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.448608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.448986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.449394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.449758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.450124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.450489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.450841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81612) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 23 22:08:36.456917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559d22a489c0 pid: 81613 (default app) Aug 23 22:08:36.460718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a483c5-9c80-4831-a394-c96a190bc7d4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:36.461115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a483c5-9c80-4831-a394-c96a190bc7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:36.461513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a483c5-9c80-4831-a394-c96a190bc7d4 None None] PATH_INFO: `/v3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:36.465994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1/1] 10.222.0.73 () {50 vars in 791 bytes} [Tue Aug 23 22:08:36 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:36.496912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x559d22a489c0 pid: 81612 (default app) Aug 23 22:08:44.618769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.619365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.619365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.620728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 2/2] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:44.630505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:44.631095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:44.631095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:44.702937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: INFO dbcounter [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] Registered counter for database keystone Aug 23 22:08:44.725618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG oslo_db.sqlalchemy.engines [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81612) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 22:08:44.732173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writer thread running {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 22:08:44.922323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81612) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 22:08:44.922654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG passlib.handlers.bcrypt [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81612) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 22:08:44.978515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:44.980107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:45.114771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1/3] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 488 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 22:08:45.648797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:45.783640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: INFO dbcounter [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] Registered counter for database keystone Aug 23 22:08:45.817546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG oslo_db.sqlalchemy.engines [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81613) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 23 22:08:45.824673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writer thread running {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 23 22:08:45.948612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1818aaeb-b1bf-4e81-89d2-6b498312d423 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:45.953035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:45.953689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:45.953974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:45.954251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:45.964079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.017617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 22:08:46.017960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a82efc-92b4-46e1-821b-45be531a5b1e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.028631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 3/4] 10.222.0.73 () {58 vars in 1324 bytes} [Tue Aug 23 22:08:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:08:46.035041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:46.050962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406fa029-f0fa-4914-b075-d990a9c65224 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:46.056167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:46.057332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:46.057624 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:46.057899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] PATH_INFO: `/v3/projects/6d681c62c63c41f3b7033ae72341d972` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:46.059361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] RBAC: Authorizing `identity:get_project(project_id=6d681c62c63c41f3b7033ae72341d972)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:46.112107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING keystone.common.rbac_enforcer.enforcer [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 23 22:08:46.112379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c36489-60a1-4313-b606-e9f814a26449 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:46.118116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 2/5] 10.222.0.73 () {56 vars in 1207 bytes} [Tue Aug 23 22:08:46 2022] GET /identity/v3/projects/6d681c62c63c41f3b7033ae72341d972 => generated 344 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:48.097646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177b02-e534-46a0-aaba-908b4603d6a2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:48.098323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177b02-e534-46a0-aaba-908b4603d6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:48.098323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01177b02-e534-46a0-aaba-908b4603d6a2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:48.103287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 4/6] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:48 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:48.109672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:48.110162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:48.110338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:48.130965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:48.132669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:48.140750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 3/7] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 23 22:08:48.422229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177b02-e534-46a0-aaba-908b4603d6a2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:48.438164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01177b02-e534-46a0-aaba-908b4603d6a2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:48.444999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3826d61-3168-4929-9493-ebb23d42cb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:48.445696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3826d61-3168-4929-9493-ebb23d42cb50 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:48.446221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3826d61-3168-4929-9493-ebb23d42cb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:48.446365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3826d61-3168-4929-9493-ebb23d42cb50 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:48.456475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3826d61-3168-4929-9493-ebb23d42cb50 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:48.458324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3826d61-3168-4929-9493-ebb23d42cb50 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:48.469281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 5/8] 10.222.0.73 () {58 vars in 1324 bytes} [Tue Aug 23 22:08:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:08:48.479300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:48.496127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717e1080-5ae4-4a10-8696-df0f572be364 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:48.503212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df94f784-0b31-4743-9a30-92b9adbc37d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:48.504295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94f784-0b31-4743-9a30-92b9adbc37d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:48.504791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94f784-0b31-4743-9a30-92b9adbc37d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:48.505325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df94f784-0b31-4743-9a30-92b9adbc37d9 admin admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:48.508512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94f784-0b31-4743-9a30-92b9adbc37d9 admin admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:48.512086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df94f784-0b31-4743-9a30-92b9adbc37d9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:48.515128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 4/9] 10.222.0.73 () {56 vars in 1198 bytes} [Tue Aug 23 22:08:48 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:50.631434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19244719-91db-4739-85e0-0ff03ecb3610 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.632218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19244719-91db-4739-85e0-0ff03ecb3610 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.632371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19244719-91db-4739-85e0-0ff03ecb3610 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.633845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 6/10] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:50.636959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e78e7-53f7-4e2f-927a-58dee1a9b128 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.637505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e78e7-53f7-4e2f-927a-58dee1a9b128 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.638004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45e78e7-53f7-4e2f-927a-58dee1a9b128 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.640992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 5/11] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:50.642223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.642585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.642967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.646487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:50.646883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:50.647282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:50.667982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:50.669391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:50.701681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG passlib.handlers.bcrypt [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=81613) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 23 22:08:50.702348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG passlib.handlers.bcrypt [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81613) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 23 22:08:50.755329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:50.757277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:50.789971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 6/12] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:50.869262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 7/13] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:51.075175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.113896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.195059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.237932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c96e63-3ddd-4be4-a4a8-69d38d17a3df None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.290466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.291467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.291938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.292408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.311778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:51.327129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:51.328718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:08:51.329773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 7/14] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:08:51.338671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.358870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4972be0c-f539-48f7-aaf9-91c3bde99c8e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.361916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.403834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.404738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.405056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.405340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.455614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.476506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53510326-994a-4087-9a77-d1de9e17b582 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.509741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.512663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.517609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.518241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.518522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.518798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.519488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:51.519797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.521252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.540111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 8/15] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:51.541632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.649557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 8/16] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:08:51.658226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.661727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.677879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59aba4-17d6-4a94-b737-7970985da959 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.723798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.724672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.725716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.726146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.728239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.730735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.733742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 9/17] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:51.742973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.766896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:51.806528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf9a948-6cf5-40f6-be43-4bc047bb33c9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.845745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.846661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.846944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.847222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.873594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b67ca5-7bb2-4b58-b84d-916d5f8bf451 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:51.875621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.877158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:08:51.902068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:51.903811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:08:51.905523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:08:51.905523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:08:51.905523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:08:51.905523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 9/18] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:08:51.914348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.923103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:51.923751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:51.924030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:51.924302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:51.925486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:51.926969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:51.928628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 10/19] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:51.934730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:51.992376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.043349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:52.056895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc34d7a-0463-4b01-971b-e43dfe232016 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.066526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4d5317-f23d-49b7-a200-716a30e00563 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:52.097799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.098435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.098719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.098996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.099664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:52.100757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.101953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00598e-adbe-41b3-8b3c-a083a2f8567b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.110984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:52.111657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:52.112046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:52.112401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:52.113280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.domain.id=default, user.id=0f6d02bd20804957aa92efbfa08393a0) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:52.113670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:52.115523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3fc4ce-36c5-4136-8492-14ccdd23691a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:52.124026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 11/20] 10.222.0.73 () {56 vars in 1322 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/role_assignments?user.id=0f6d02bd20804957aa92efbfa08393a0&scope.domain.id=default&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 233 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:52.133637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 10/21] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:08:54.363038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a32b-b285-4a2b-9ddf-74eb6b627ecf None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.364063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a32b-b285-4a2b-9ddf-74eb6b627ecf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.364541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa9a32b-b285-4a2b-9ddf-74eb6b627ecf None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.366707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 12/22] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:54.372644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4bb0e-dffe-4fd2-889d-da30655321a3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.373571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4bb0e-dffe-4fd2-889d-da30655321a3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.374060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4bb0e-dffe-4fd2-889d-da30655321a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.402136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0dc4bb0e-dffe-4fd2-889d-da30655321a3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:54.403704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0dc4bb0e-dffe-4fd2-889d-da30655321a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:54.505435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 11/23] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:54.714033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbd661-97fe-433a-b244-a77944531723 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.714732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbd661-97fe-433a-b244-a77944531723 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.715202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cfbd661-97fe-433a-b244-a77944531723 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.716788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 13/24] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:08:54.722895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc53c75-e279-4871-b3e2-b055d7c05c90 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.723500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc53c75-e279-4871-b3e2-b055d7c05c90 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.723966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc53c75-e279-4871-b3e2-b055d7c05c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.753340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5fc53c75-e279-4871-b3e2-b055d7c05c90 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:54.755667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5fc53c75-e279-4871-b3e2-b055d7c05c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:54.759259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a1fa9-b097-4519-9033-dc7e41e62ba0 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.760160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a1fa9-b097-4519-9033-dc7e41e62ba0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.761001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06a1fa9-b097-4519-9033-dc7e41e62ba0 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.764086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 14/25] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:54.786094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59211a-21fc-4a99-9cc6-64b9de144f4e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:54.787037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59211a-21fc-4a99-9cc6-64b9de144f4e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:54.788732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c59211a-21fc-4a99-9cc6-64b9de144f4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:54.837553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:54.850186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5c59211a-21fc-4a99-9cc6-64b9de144f4e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:54.851438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c59211a-21fc-4a99-9cc6-64b9de144f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:54.883400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 12/26] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:54.969304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 15/27] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:55.105624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a282d-b6e1-47a5-b5e5-a668c1a03537 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.106335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a282d-b6e1-47a5-b5e5-a668c1a03537 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.106811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207a282d-b6e1-47a5-b5e5-a668c1a03537 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.108366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 13/28] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:55.114542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.115210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.116768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.140485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:55.142105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:55.184196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a282d-b6e1-47a5-b5e5-a668c1a03537 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.260120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 16/29] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:08:55.354774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207a282d-b6e1-47a5-b5e5-a668c1a03537 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.370766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.375556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.401273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.402017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.402215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.402495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.426697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:55.433380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:55.434265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:08:55.435474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 14/30] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:08:55.436287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.510710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.524110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30b18de-0025-4501-a556-4914f70fd81d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.568268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.569557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.570036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.570517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.581920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.583729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:08:55.603691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:55.604582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:08:55.605448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:08:55.605448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:08:55.605448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:08:55.607285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 17/31] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:08:55.608473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.630049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae379767-295b-4938-923e-5d4260c0407a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.675205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.694914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.696194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.696814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.697640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.699056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.701990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.731153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ffff5-670b-4707-8b86-5439c6def308 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.777484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.778015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.778300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.778582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.779280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:55.779589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.781178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:55.798343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 18/32] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:55.798343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.929743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eefa33-b1b6-4ba6-8bd8-c62ead0d2b3a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:55.941346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 15/33] 10.222.0.73 () {60 vars in 1131 bytes} [Tue Aug 23 22:08:55 2022] POST /identity/v3/roles => generated 219 bytes in 506 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:55.942514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:55.952787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:55.975195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:55.976329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:55.976757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:55.977213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:55.977971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:55.979443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:56.105741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:56.105741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dcaa3f1-cb66-4490-8435-b56d2f6a8eb8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:56.112632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:56.113419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.113709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.113991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.114680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:08:56.114989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:56.117290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:56.132209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 16/34] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:08:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:56.133360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:56.332883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 19/35] 10.222.0.73 () {60 vars in 1131 bytes} [Tue Aug 23 22:08:55 2022] POST /identity/v3/roles => generated 225 bytes in 539 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:56.333855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:56.339525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a4ed57-7074-4e52-bf37-a917af1472f8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:56.411742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:56.421305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.421788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.422017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.423973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:56.425555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:56.432338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 17/36] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:08:56 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:56.433981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:56.473873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7794ac-608f-4386-8e1a-76ead324c91f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:56.493557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:56.532333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463254a8-3886-4cd2-94ec-1e01d14e3373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:56.532977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463254a8-3886-4cd2-94ec-1e01d14e3373 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.533434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463254a8-3886-4cd2-94ec-1e01d14e3373 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.533772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463254a8-3886-4cd2-94ec-1e01d14e3373 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.534391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463254a8-3886-4cd2-94ec-1e01d14e3373 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:56.536039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463254a8-3886-4cd2-94ec-1e01d14e3373 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:56.620139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65aa9761-bbab-4c9c-9fc4-13283611a8e8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:56.665601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7591a0-631b-4ae5-acae-b9b019ed2d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:56.666544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7591a0-631b-4ae5-acae-b9b019ed2d5e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.667114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7591a0-631b-4ae5-acae-b9b019ed2d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.667614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7591a0-631b-4ae5-acae-b9b019ed2d5e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.669702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7591a0-631b-4ae5-acae-b9b019ed2d5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:56.672399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7591a0-631b-4ae5-acae-b9b019ed2d5e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:56.675317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 18/37] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:56.677937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c8ae0-b2ca-42bc-b433-3b35289996b3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.678481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c8ae0-b2ca-42bc-b433-3b35289996b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.678960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272c8ae0-b2ca-42bc-b433-3b35289996b3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.682735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 19/38] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:56 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:56.682735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c8ae0-b2ca-42bc-b433-3b35289996b3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:56.695048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 20/39] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:08:56 2022] POST /identity/v3/projects => generated 301 bytes in 362 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:56.697413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c920-eff4-4e27-9ae6-e9426dbec408 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.697894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c920-eff4-4e27-9ae6-e9426dbec408 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.698296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c66c920-eff4-4e27-9ae6-e9426dbec408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.724171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6c66c920-eff4-4e27-9ae6-e9426dbec408 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:56.726818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c66c920-eff4-4e27-9ae6-e9426dbec408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:56.780290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:56.878916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 21/40] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:56.881812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:56.887877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc2747-a614-4982-b94f-989af2e7254d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.888614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc2747-a614-4982-b94f-989af2e7254d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.889278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dc2747-a614-4982-b94f-989af2e7254d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.892270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 22/41] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:56 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:56.902380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.902876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.903368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.905357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272c8ae0-b2ca-42bc-b433-3b35289996b3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:56.939672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:56.941170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:56.978779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:56.979934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:56.980439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:56.980936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] PATH_INFO: `/v3/domains/default/users/0f6d02bd20804957aa92efbfa08393a0/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:56.984614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=0f6d02bd20804957aa92efbfa08393a0, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:56.987981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:57.121666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 20/42] 10.222.0.73 () {58 vars in 1361 bytes} [Tue Aug 23 22:08:56 2022] PUT /identity/v3/domains/default/users/0f6d02bd20804957aa92efbfa08393a0/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:08:57.192331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.218765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 23/43] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:57.357795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9b024e-9b40-47ce-8e7e-0bb3a0044837 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.398212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.398910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.399201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.399482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.400184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:57.401619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:57.478416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 21/44] 10.222.0.73 () {60 vars in 1131 bytes} [Tue Aug 23 22:08:57 2022] POST /identity/v3/roles => generated 223 bytes in 287 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:08:57.513419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.678053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:57.695706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71216bdb-40f9-43e1-9052-977cff993724 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.743299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9234a64-22bc-46b9-8567-8f7e1e6ab3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.744327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9234a64-22bc-46b9-8567-8f7e1e6ab3db None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.744605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9234a64-22bc-46b9-8567-8f7e1e6ab3db None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.744887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9234a64-22bc-46b9-8567-8f7e1e6ab3db None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.746129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9234a64-22bc-46b9-8567-8f7e1e6ab3db None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:57.747654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9234a64-22bc-46b9-8567-8f7e1e6ab3db None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:57.749578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 24/45] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:08:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:08:57.759654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:57.780991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:57.882059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4604079c-e50d-43e6-829c-bfa1795ea805 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:57.924466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521c2175-0c71-43f8-b193-368403ce08a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:57.925281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c2175-0c71-43f8-b193-368403ce08a4 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:57.925677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c2175-0c71-43f8-b193-368403ce08a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:57.926117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521c2175-0c71-43f8-b193-368403ce08a4 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:57.926907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c2175-0c71-43f8-b193-368403ce08a4 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:08:57.929234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521c2175-0c71-43f8-b193-368403ce08a4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:08:58.027133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 22/46] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:08:57 2022] POST /identity/v3/projects => generated 312 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:08:59.229535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab89a-13a1-4578-8810-45f44db90c13 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.230310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab89a-13a1-4578-8810-45f44db90c13 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.230496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ab89a-13a1-4578-8810-45f44db90c13 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.233128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 25/47] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:08:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:08:59.238962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.239765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.239765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.263095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:08:59.266081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:08:59.371056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:08:59.384627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 23/48] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:08:59.655882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab89a-13a1-4578-8810-45f44db90c13 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:08:59.794669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ab89a-13a1-4578-8810-45f44db90c13 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:08:59.833642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:08:59.834839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:08:59.835055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:08:59.835329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:08:59.847073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:08:59.852497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:08:59.854184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:08:59.855590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 26/49] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:08:59.860399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.039908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2f71bd-c5fe-4835-ac10-f859a636b48e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.101274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.102228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.102723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.103210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.104425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:00.104964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.107586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cbb0c-d9b7-4a86-9e58-5ef08f75af82 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.130463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 24/50] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:00.136760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.149001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ef0bc-6546-42a8-b6bd-cf4d44326b57 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.149535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ef0bc-6546-42a8-b6bd-cf4d44326b57 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.150023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05ef0bc-6546-42a8-b6bd-cf4d44326b57 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.152535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 25/51] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:00.159642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.160824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.161139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.161416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.180991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:00.182320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:00.263770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa89bce3-fdc3-45cb-a681-f4a6610d0085 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.268292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.298934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 26/52] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:00.305318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.306075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.306406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.306685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.307910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.309489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.311147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 27/53] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.320283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.565819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb2c3d-fd20-4081-893c-523f71b1653b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.581697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.610182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.610835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.611119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.611398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.612547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.614044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.619585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 27/54] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.626265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.686749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.702873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36683b9c-3ce6-4a56-a51b-6203a143d840 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.726272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:00.752660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.753771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.754277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.754766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.756722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.759553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.762903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 28/55] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:00.769988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:00.778109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a37440-80af-4c4d-b042-8bf5387cbecc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.826087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.826623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.826908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.827184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.827950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.domain.id=default, user.id=0f6d02bd20804957aa92efbfa08393a0) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:00.828249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.829754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa48190-19c9-4c42-b438-032afb9f923a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:00.840149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 28/56] 10.222.0.73 () {56 vars in 1322 bytes} [Tue Aug 23 22:09:00 2022] GET /identity/v3/role_assignments?user.id=0f6d02bd20804957aa92efbfa08393a0&scope.domain.id=default&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 530 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:00.901830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20400004-8af8-4eb9-9a4b-175d2f8f703f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:00.943093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f9c567-f180-4612-8674-8cf72daa1277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:00.944275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9c567-f180-4612-8674-8cf72daa1277 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:00.944556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9c567-f180-4612-8674-8cf72daa1277 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:00.944829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f9c567-f180-4612-8674-8cf72daa1277 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:00.945570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9c567-f180-4612-8674-8cf72daa1277 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:00.947115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f9c567-f180-4612-8674-8cf72daa1277 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:01.061766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 29/57] 10.222.0.73 () {60 vars in 1140 bytes} [Tue Aug 23 22:09:00 2022] POST /identity/v3/projects => generated 298 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:09:03.064232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e127c-880c-4b10-a6b9-5b24afea7926 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.064807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e127c-880c-4b10-a6b9-5b24afea7926 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.064807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102e127c-880c-4b10-a6b9-5b24afea7926 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.066183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 29/58] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:03.072530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.073275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.073275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.086341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.092025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:03.093312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:03.167291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 30/59] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:03.444497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e127c-880c-4b10-a6b9-5b24afea7926 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:03.569952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102e127c-880c-4b10-a6b9-5b24afea7926 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:03.592944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.619653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0ba62a-8d1c-49ac-b7e6-0d4089951f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:03.620546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ba62a-8d1c-49ac-b7e6-0d4089951f9d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.621173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ba62a-8d1c-49ac-b7e6-0d4089951f9d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.621640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0ba62a-8d1c-49ac-b7e6-0d4089951f9d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.623910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ba62a-8d1c-49ac-b7e6-0d4089951f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:03.626640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0ba62a-8d1c-49ac-b7e6-0d4089951f9d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:03.628648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 30/60] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:03.636361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:03.745786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b534eb1-01a6-4d6e-a647-2f16d96dc098 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:03.783062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0df5e734-2cba-4271-aa39-1d815c97f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:03.783700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e734-2cba-4271-aa39-1d815c97f2cb None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:03.783988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e734-2cba-4271-aa39-1d815c97f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:03.784263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df5e734-2cba-4271-aa39-1d815c97f2cb None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:03.784979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5e734-2cba-4271-aa39-1d815c97f2cb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:03.786474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0df5e734-2cba-4271-aa39-1d815c97f2cb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:03.799495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:03.899262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 31/61] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:03 2022] POST /identity/v3/users => generated 275 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:09:07.569735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3393a1a-268d-4f8a-98f7-7298c5353354 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:07.571271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3393a1a-268d-4f8a-98f7-7298c5353354 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:07.572057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3393a1a-268d-4f8a-98f7-7298c5353354 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:07.574916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 31/62] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:07 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:07.582068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1b52e-f15f-4018-b859-1fc8830a0546 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:07.584034 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1b52e-f15f-4018-b859-1fc8830a0546 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:07.584805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1b52e-f15f-4018-b859-1fc8830a0546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:07.634878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-def1b52e-f15f-4018-b859-1fc8830a0546 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:07.637442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-def1b52e-f15f-4018-b859-1fc8830a0546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:07.787814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 32/63] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:08.038522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8abd67-c353-4b1d-9acc-bfb6cb03958e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.039322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8abd67-c353-4b1d-9acc-bfb6cb03958e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.040002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8abd67-c353-4b1d-9acc-bfb6cb03958e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.043366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 32/64] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:08 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:08.048779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765592a-1eee-4df5-a708-db9f0353abba None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.049096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765592a-1eee-4df5-a708-db9f0353abba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.049384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6765592a-1eee-4df5-a708-db9f0353abba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.078204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6765592a-1eee-4df5-a708-db9f0353abba None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:08.079447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6765592a-1eee-4df5-a708-db9f0353abba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:08.174814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 33/65] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:08.280840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8abd67-c353-4b1d-9acc-bfb6cb03958e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:08.356789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437130dd-d154-4c69-82d7-60ab7f784ba8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.357120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437130dd-d154-4c69-82d7-60ab7f784ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.357297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437130dd-d154-4c69-82d7-60ab7f784ba8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.359037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 34/66] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:08.369044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a7a1e-9e80-4806-837e-f1045e158a14 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.369666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a7a1e-9e80-4806-837e-f1045e158a14 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.370345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3a7a1e-9e80-4806-837e-f1045e158a14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.392307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8e3a7a1e-9e80-4806-837e-f1045e158a14 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:08.394695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8e3a7a1e-9e80-4806-837e-f1045e158a14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:08.429988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8abd67-c353-4b1d-9acc-bfb6cb03958e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:08.457569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:08.485345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 35/67] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:08.487047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1387ece-2b1d-409a-9e3f-44f4204d522a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.487580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1387ece-2b1d-409a-9e3f-44f4204d522a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.488085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1387ece-2b1d-409a-9e3f-44f4204d522a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.490916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 36/68] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:08.500083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.500695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.501368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.524271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:08.526012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:08.531662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:08.532757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.533301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.533796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.590119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:08.608301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:08.609411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:08.610180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:08.611251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 37/69] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:08.611690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 33/70] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:08.630122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:08.695589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:08.789600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:08.790501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdaa0c-8732-4e66-86a8-397eee161f29 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:08.809305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:08.837966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:08.839347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.839800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.840259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.841359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:08.841880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:08.844254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa8a50c-8cd6-458b-b9a1-45228b0c2256 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:08.859071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 38/71] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:08.860669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c450f0-0f2d-4fe3-a529-52ece4112bb2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.860965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c450f0-0f2d-4fe3-a529-52ece4112bb2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.861279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c450f0-0f2d-4fe3-a529-52ece4112bb2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.862827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 39/72] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:08.863375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c450f0-0f2d-4fe3-a529-52ece4112bb2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:08.869939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a0c656-5e52-478e-892c-0bc11ba849c6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:08.915967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:08.916953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:08.917437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:08.917850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:08.931092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:08.936016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:08.937694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 34/73] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:08.939200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.012619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c450f0-0f2d-4fe3-a529-52ece4112bb2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.059862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14853ce6-6303-4906-8f31-4d530fe96bfe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.063179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89d1f58-eb6e-4f75-a342-1ff9ed45b11b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.064102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d1f58-eb6e-4f75-a342-1ff9ed45b11b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.064381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d1f58-eb6e-4f75-a342-1ff9ed45b11b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.064931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89d1f58-eb6e-4f75-a342-1ff9ed45b11b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.071268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.079048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89d1f58-eb6e-4f75-a342-1ff9ed45b11b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e89d1f58-eb6e-4f75-a342-1ff9ed45b11b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:09.085167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:09.086899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 40/74] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:09.088342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.089683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.089683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.109749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:09.109932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.110591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.111051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.111491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.111575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:09.134552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:09.136049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:09.138105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 35/75] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:08 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:09.138589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.158909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.217566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 41/76] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:09.226831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.263840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b113d48-4739-47a0-b2c3-d28bde360024 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.313840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.314596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.315375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.316421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.318068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:09.319807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:09.322530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:09.336721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 36/77] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:09.337394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.376987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.377751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8799207b-d06c-4815-b6f2-ac35810035d0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.426568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.427295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.427637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.428146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.439724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:09.445920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:09.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:09.449124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 42/78] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:09.450375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.481689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933cb8f-8e0f-488e-b00c-f69bebb1dae1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.487646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.526928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b10348-5b02-4453-911a-8625e977932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.527654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b10348-5b02-4453-911a-8625e977932a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.528001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b10348-5b02-4453-911a-8625e977932a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.528362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b10348-5b02-4453-911a-8625e977932a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.529264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b10348-5b02-4453-911a-8625e977932a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:09.529862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b10348-5b02-4453-911a-8625e977932a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:09.532518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b10348-5b02-4453-911a-8625e977932a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:09.545579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 37/79] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:09.546619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b10348-5b02-4453-911a-8625e977932a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.591812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f6459-a115-4623-9b86-9e4f4ab9d5fa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.631736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.632762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.632762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.632954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.667123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.680455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b10348-5b02-4453-911a-8625e977932a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.710295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:09.711829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:09.719353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.719997 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.720278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.720551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.721864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:09.723329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:09.725742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 38/80] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:09.725878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.745608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.831158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 43/81] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:09.832017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.833275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ca76ef-c770-4fc2-9acc-4bd1b8954060 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.875206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:09.875852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:09.876136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:09.876551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:09.877456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:09.878040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:09.879419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:09.888774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 39/82] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:09.889619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:09.910928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:09.965995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a349891a-03d1-455c-9881-e20ede3d0dd3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:09.993330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.005877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.006529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.006808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.007122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.008423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.009985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.011740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 44/83] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:10.012625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.020448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609fa4ce-261c-4a81-b1cd-e17b9ab1e2f0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.065197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.065842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.066123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.066398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.067552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.069571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.071201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 40/84] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:09 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:10.071993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.144554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caac4ad-a701-4353-8e7e-52565cd5aaa7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.167071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.184700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636283bb-47df-4d54-a36d-3f7b3a8ee3f0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.203747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.204509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.204891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.205205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.224488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.225599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.225894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.226308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.227675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.229904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.237280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 41/85] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:10.237280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.282408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.293855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.295377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.363467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bba50a-f7c6-4749-977a-5bcaa78855a0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.402478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.403048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.403338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.403612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.405409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.408042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 45/86] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:10.408238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.408363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.408755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 42/87] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:10.417382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.443530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.526309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.535625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054dce-0f80-4d79-b686-41193894bbb8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.537577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e10f7a-efe9-477a-ba6a-bfea9e9197f4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.581881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.582621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.583005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.583897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.584675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.586368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.586563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.586991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.587281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.587552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.660062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.662069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.675500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 46/88] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:10 2022] POST /identity/v3/users => generated 289 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:10.676534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.696702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:10.830838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791c0d8b-36f1-41ed-8847-724f3b069e1f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:10.852169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 43/89] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:10.854568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.897489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:10.898471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:10.898930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:10.899412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:10.900800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=0a614963f72e4c8abe5f162fabb45136) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:10.901337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:10.903886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:10.915137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 47/90] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/role_assignments?user.id=0a614963f72e4c8abe5f162fabb45136&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:10.916375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:10.947944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.013241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89f0ece3-c97a-41a1-9c8b-d17ccefcf2f5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.070679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.098570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.100548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.103911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.103911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.103911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.105619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.108422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 44/91] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.110436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.125339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b17420-69ef-4c2d-a8c4-764c3d20e443 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.175534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.175534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.175534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.175534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.290342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebafea6e-d082-45d3-b220-ffc1b90d0ead None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.330070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.333355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.338496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.339155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.339444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.339720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.341318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.342810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.344555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 45/92] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.345577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.370689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.463314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.545487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c990f1-9ca0-4b08-ac12-2407db6e49ba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.549326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 48/93] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:11.550732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.637874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.638505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.638779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.639043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.639789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=0f6d02bd20804957aa92efbfa08393a0) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:11.640095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.641596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.656312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 46/94] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/role_assignments?user.id=0f6d02bd20804957aa92efbfa08393a0&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.656908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:11.736225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd66a70-9889-400a-bba5-45eba5a1eb65 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.776411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:11.842856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:11.844609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:11.845458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:11.846250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:11.848058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] RBAC: Adding query filter params (role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=0a614963f72e4c8abe5f162fabb45136) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:11.850072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:11.853533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:11.874862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 49/95] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/role_assignments?user.id=0a614963f72e4c8abe5f162fabb45136&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 259 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:11.947109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee56002-adc6-49c2-b968-08cfa16666d6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:11.997890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:12.021720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ff4cbe-1446-4693-90c7-5bf41e9452bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.025042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff4cbe-1446-4693-90c7-5bf41e9452bf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.025042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff4cbe-1446-4693-90c7-5bf41e9452bf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.025042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ff4cbe-1446-4693-90c7-5bf41e9452bf None admin] PATH_INFO: `/v3/projects/4aa12113285c469683047af9078a81ba` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.027045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff4cbe-1446-4693-90c7-5bf41e9452bf None admin] RBAC: Authorizing `identity:get_project(project_id=4aa12113285c469683047af9078a81ba)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:12.030368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ff4cbe-1446-4693-90c7-5bf41e9452bf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:12.033876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 47/96] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:11 2022] GET /identity/v3/projects/4aa12113285c469683047af9078a81ba => generated 312 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:12.041349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:12.261519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9536c92-6af0-4a9b-8497-8d04ed7e7990 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:12.332069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:12.336615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:12.337581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:12.338420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:12.340195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=4aa12113285c469683047af9078a81ba, user.id=0a614963f72e4c8abe5f162fabb45136) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:12.341136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:12.344555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68d806-145b-4d84-822d-c3c5016450aa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:12.356799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 50/97] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:12 2022] GET /identity/v3/role_assignments?user.id=0a614963f72e4c8abe5f162fabb45136&scope.project.id=4aa12113285c469683047af9078a81ba&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:13.632334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a905f-a921-4efe-9c73-aba22c904630 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.632891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a905f-a921-4efe-9c73-aba22c904630 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.633328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25a905f-a921-4efe-9c73-aba22c904630 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.636006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 48/98] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:13.642955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5fd04-ef7d-4c21-ac7f-134058ca64f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.644109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5fd04-ef7d-4c21-ac7f-134058ca64f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.644606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f5fd04-ef7d-4c21-ac7f-134058ca64f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.671524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b4f5fd04-ef7d-4c21-ac7f-134058ca64f4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:13.673101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b4f5fd04-ef7d-4c21-ac7f-134058ca64f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:13.721075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:13.769360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 51/99] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:13.824261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9e67e-e50d-4b97-a3b2-f8b1647f7b27 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.825196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9e67e-e50d-4b97-a3b2-f8b1647f7b27 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.825940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b9e67e-e50d-4b97-a3b2-f8b1647f7b27 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.829125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 49/100] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:13.834462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:13.834741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:13.835238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:13.859132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:13.860707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:13.955616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 52/101] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:14.180571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9e67e-e50d-4b97-a3b2-f8b1647f7b27 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.364475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.418616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b9e67e-e50d-4b97-a3b2-f8b1647f7b27 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.511342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.512015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.515086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.515437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.547669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:14.560578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:14.561528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:14.562310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:14.565103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 50/102] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/roles/reader => generated 84 bytes in 387 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:14.571719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.603865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.607903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cba45-50ef-48ec-9157-b5f40db64c28 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.657580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:14.664613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.666081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.666534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.667045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.669758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.671566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.675234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 53/103] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:14.696672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.768742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f946755-5bc9-4ce9-9bd4-501a67d7091f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:14.853651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:14.854278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.854562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.854847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.855533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:14.855841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:14.857284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac12695-d76f-4934-88be-06baf00dae78 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:14.880901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 51/104] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:14.882360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639c340-551f-4865-b522-5b42847c2ef5 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:14.882659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639c340-551f-4865-b522-5b42847c2ef5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:14.882938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e639c340-551f-4865-b522-5b42847c2ef5 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:14.884256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 52/105] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:14.889819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639c340-551f-4865-b522-5b42847c2ef5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:14.949256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2248227e-cce9-4e5b-9bf9-ff2079181fd6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.019675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f9a2ac-80ae-4e33-be9b-d50cf8ded68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.020222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a2ac-80ae-4e33-be9b-d50cf8ded68e None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.020510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a2ac-80ae-4e33-be9b-d50cf8ded68e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.021395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f9a2ac-80ae-4e33-be9b-d50cf8ded68e None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.022080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9a2ac-80ae-4e33-be9b-d50cf8ded68e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.023801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f9a2ac-80ae-4e33-be9b-d50cf8ded68e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.079214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e639c340-551f-4865-b522-5b42847c2ef5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.095094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.119316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.120013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.120544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.123377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] PATH_INFO: `/v3/users/3468051f65e64a1295f6246a758b8de7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.129434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 54/106] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:09:14 2022] POST /identity/v3/projects => generated 302 bytes in 433 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:15.131499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fc0a-d032-406e-ac65-c3a0698fd8e2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.134786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fc0a-d032-406e-ac65-c3a0698fd8e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.134786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb4fc0a-d032-406e-ac65-c3a0698fd8e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.153135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] RBAC: Authorizing `identity:get_user(user_id=3468051f65e64a1295f6246a758b8de7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.154645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.156499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 53/107] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:14 2022] GET /identity/v3/users/3468051f65e64a1295f6246a758b8de7 => generated 289 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:15.166749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.174514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-acb4fc0a-d032-406e-ac65-c3a0698fd8e2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:15.176544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-acb4fc0a-d032-406e-ac65-c3a0698fd8e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:15.212648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.305951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3f0cf-6a17-4278-b55b-6b85cf4d7e08 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.320064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 55/108] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:15.369599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.370590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.371060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.371513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.459359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea3755-8f62-4545-8c2a-11b83d6e00c3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.459623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea3755-8f62-4545-8c2a-11b83d6e00c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.459921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea3755-8f62-4545-8c2a-11b83d6e00c3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.465566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 56/109] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:15 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:15.469862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.471418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.471707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.471989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:15.508828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:15.512445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:15.570723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:15.573435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:15.669294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 57/110] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:15.756420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 54/111] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 590 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:09:15.765091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.779065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:15.877604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.915473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493e6d94-d741-4fe8-a86e-61e8c238687e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:15.956957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:15.987573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c404b81-4d2a-4711-9f97-bf56e7c1df57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:15.990706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c404b81-4d2a-4711-9f97-bf56e7c1df57 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:15.991295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c404b81-4d2a-4711-9f97-bf56e7c1df57 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:15.991668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c404b81-4d2a-4711-9f97-bf56e7c1df57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.000995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21752757-7958-41ac-8040-e6038fa0e814 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.022246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c404b81-4d2a-4711-9f97-bf56e7c1df57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8c404b81-4d2a-4711-9f97-bf56e7c1df57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:16.027204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:16.028093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:16.028852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 55/112] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:16.033705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b621c-8f20-4e78-859f-b5865480689d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.034687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b621c-8f20-4e78-859f-b5865480689d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.035064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2b621c-8f20-4e78-859f-b5865480689d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.036622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 56/113] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:16.037801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b621c-8f20-4e78-859f-b5865480689d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.066125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.066800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.067084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.067365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.069759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.071113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.077056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 58/114] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:15 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.077056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.242172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2b621c-8f20-4e78-859f-b5865480689d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.244751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9de82-75eb-458f-9d9e-fabf40e21054 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.277794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:16.284768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e7fd1-6ac0-4944-8dfd-ca6e6eaa45ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.285461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e7fd1-6ac0-4944-8dfd-ca6e6eaa45ce None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.285744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e7fd1-6ac0-4944-8dfd-ca6e6eaa45ce None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.286017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e7fd1-6ac0-4944-8dfd-ca6e6eaa45ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.299934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e7fd1-6ac0-4944-8dfd-ca6e6eaa45ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-718e7fd1-6ac0-4944-8dfd-ca6e6eaa45ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:16.306807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:16.307727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:16.308490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:16.310071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 57/115] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:16.310751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a1907-b047-4e7a-9ad8-144bb0afc6fc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.311072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a1907-b047-4e7a-9ad8-144bb0afc6fc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.311310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a1907-b047-4e7a-9ad8-144bb0afc6fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.314969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.315967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.316588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.317251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.318378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:16.319324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.321892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.332030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2c4a1907-b047-4e7a-9ad8-144bb0afc6fc None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:16.332420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 59/116] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.334109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.338746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2c4a1907-b047-4e7a-9ad8-144bb0afc6fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:16.353353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:16.416294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 58/117] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:16.428927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c353d-ba7d-406a-9ca7-eca950647d1e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.429804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c353d-ba7d-406a-9ca7-eca950647d1e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.430920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94c353d-ba7d-406a-9ca7-eca950647d1e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.435396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 59/118] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:16 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:16.437331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c353d-ba7d-406a-9ca7-eca950647d1e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.480439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a033fa61-d87a-4197-99be-ef2936fae40f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.548102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.548712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.549103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.549483 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.550299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] RBAC: Adding query filter params (role.id=233bfd8b4640483288d9822efc76cb09, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=3468051f65e64a1295f6246a758b8de7) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:16.555783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.557538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.564894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 60/119] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/role_assignments?user.id=3468051f65e64a1295f6246a758b8de7&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=233bfd8b4640483288d9822efc76cb09 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.565925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.669649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:16.680306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94c353d-ba7d-406a-9ca7-eca950647d1e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.719565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:16.744605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2825ed61-3ea9-4da1-817e-d8df5e0a39e6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:16.756883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.757668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.757795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.758144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.758856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:16.759161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.762437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05830d7-712b-492d-a18b-44e4a8f67421 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.792907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 60/120] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.795126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.795618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.796118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.818365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:16.819967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:16.820786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:16.821522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:16.824188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:16.827214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:16.830502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 61/121] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:16.832297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:16.861226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:16.863404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:16.986992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 61/122] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:16.993030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.080394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ead7e94-13d5-4304-b8b8-02223307396d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.169049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.170618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.171371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.172137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.176327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:17.180028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:17.185627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 62/123] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:17.190412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.229654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.272503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99bdb27-d98f-4174-893c-10739632bd36 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.322011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.383680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.384307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.384591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.384860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.386100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:17.387602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:17.389678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 62/124] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:16 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:17.390496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.476315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7620df86-d919-44f1-9235-7558013f1f33 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.528552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.529234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.529783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.530067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.535919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a8e0c6-5b22-4c5b-9a6c-d5788aa3a9d5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.545230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:17.558436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:17.559307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:17.560052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:17.565084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 63/125] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:17.567063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.607317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.608895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.609748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.610543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] PATH_INFO: `/v3/projects/4aa12113285c469683047af9078a81ba` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.612966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] RBAC: Authorizing `identity:get_project(project_id=4aa12113285c469683047af9078a81ba)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:17.616865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:17.620570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 63/126] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/projects/4aa12113285c469683047af9078a81ba => generated 312 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:17.622499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.672804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.723271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff88342-e227-4977-9ab5-8c451aabf51d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.732160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:17.799718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.801515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.802364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.803153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/0a614963f72e4c8abe5f162fabb45136/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.807286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bd3cad3b8f49989950c0a9e6f1976c, user_id=0a614963f72e4c8abe5f162fabb45136, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:17.810606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:17.862859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b569e-528e-4a96-901d-0abea1fbe190 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:17.908946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 64/127] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:17 2022] PUT /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/0a614963f72e4c8abe5f162fabb45136/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:17.911250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:17.957898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:17.958720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:17.959041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:17.959294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:17.986144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:17.991775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:17.993940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:17.995458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:18.001512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 64/128] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:18.006355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.186754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ebb76-a1ff-49eb-a813-27b02c905f9c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.255505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.264156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.265121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.265571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.267085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.267085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:18.267085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.269225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.282940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 65/129] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:18.284341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.320356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70183277-8e03-4e1f-94c6-d3c9e69466a7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.333137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.363368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.364334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.364652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.364980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] PATH_INFO: `/v3/projects/4aa12113285c469683047af9078a81ba/users/0a614963f72e4c8abe5f162fabb45136/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.367239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] RBAC: Authorizing `identity:create_grant(project_id=4aa12113285c469683047af9078a81ba, user_id=0a614963f72e4c8abe5f162fabb45136, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.368740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.450635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48ea7a-62a1-42bc-8eee-27d0eb14d2b2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.497043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.497043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.497043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.497043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.497406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 65/130] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:18 2022] PUT /identity/v3/projects/4aa12113285c469683047af9078a81ba/users/0a614963f72e4c8abe5f162fabb45136/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:18.500521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:18.500521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.500713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.500783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.507685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 66/131] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:18.513529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.653691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.745187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c325a0-18d7-4251-ae2b-d6d42d87fde0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.765654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cf40f9-0755-457e-a299-90eeb8be570c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:18.810124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87042145-8230-4037-9228-38be25aa6409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.811161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87042145-8230-4037-9228-38be25aa6409 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.811652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87042145-8230-4037-9228-38be25aa6409 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.812122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87042145-8230-4037-9228-38be25aa6409 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.813635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87042145-8230-4037-9228-38be25aa6409 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.815517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87042145-8230-4037-9228-38be25aa6409 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.819797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 67/132] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:18.821497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863387-04b0-4c49-80cf-52b7c47d2d94 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.822008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863387-04b0-4c49-80cf-52b7c47d2d94 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.822473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b863387-04b0-4c49-80cf-52b7c47d2d94 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.824015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 68/133] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:18.830937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.831578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.832079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.838907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:18.840056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:18.840615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:18.841363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:18.843441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:18.846222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:18.849796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 66/134] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:18.852053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:18.858720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:18.860295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:18.926469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:18.987062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 69/135] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:18.988761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.082013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7f13b8-d273-4ddc-89ad-0090d3ff08ab None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.120385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.161956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.163808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.164323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.164818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.171232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.174368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.178188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 67/136] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:19.186098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.274480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495169db-fe5b-4037-93c8-68df369cef1a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.377692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.378280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.378570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.378999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.384774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.386332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.388130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 70/137] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:18 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 400 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:19.394012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.526743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a724d6d4-33be-4b7e-88e3-6bb8aed0c668 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.550053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.641043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e4a8c3-01cc-4c79-ae4c-b7d162cbb247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.642418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a8c3-01cc-4c79-ae4c-b7d162cbb247 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.643128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a8c3-01cc-4c79-ae4c-b7d162cbb247 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.643619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4a8c3-01cc-4c79-ae4c-b7d162cbb247 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/0a614963f72e4c8abe5f162fabb45136/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.648569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4a8c3-01cc-4c79-ae4c-b7d162cbb247 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bd3cad3b8f49989950c0a9e6f1976c, user_id=0a614963f72e4c8abe5f162fabb45136, role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.652104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258aee1f-3364-4ed1-b550-5b158d3e19df None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:19.652345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e4a8c3-01cc-4c79-ae4c-b7d162cbb247 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.679241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:19.696157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0379248b-9449-482c-8f53-e4654b88f9bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:19.697127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379248b-9449-482c-8f53-e4654b88f9bb None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.697523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379248b-9449-482c-8f53-e4654b88f9bb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.697933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0379248b-9449-482c-8f53-e4654b88f9bb None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/0f6d02bd20804957aa92efbfa08393a0/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.700230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379248b-9449-482c-8f53-e4654b88f9bb None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bd3cad3b8f49989950c0a9e6f1976c, user_id=0f6d02bd20804957aa92efbfa08393a0, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:19.701936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0379248b-9449-482c-8f53-e4654b88f9bb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:19.733121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 68/138] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:19 2022] PUT /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/0a614963f72e4c8abe5f162fabb45136/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 0 bytes in 543 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:19.733121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64624-da89-4def-8151-413592954adc None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.733121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64624-da89-4def-8151-413592954adc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.733121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f64624-da89-4def-8151-413592954adc None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.734064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 69/139] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:19.735429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64624-da89-4def-8151-413592954adc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:19.816230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 71/140] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:19 2022] PUT /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/0f6d02bd20804957aa92efbfa08393a0/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:19.818927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:19.819395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:19.819861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:19.883390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:19.885387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:20.063587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f64624-da89-4def-8151-413592954adc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:20.110993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 72/141] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:20.140913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:20.142587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.143340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.144144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.146736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:20.150063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:20.153276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 70/142] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 419 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:20.158837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:20.413166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26377e9f-d7f4-4f72-95bf-dc5a529cbf0d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:20.431084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:20.462979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:20.471237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b2b614-1903-45f9-a4e9-050fc3b495dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:20.472051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b614-1903-45f9-a4e9-050fc3b495dc None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.472277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b614-1903-45f9-a4e9-050fc3b495dc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.475780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b2b614-1903-45f9-a4e9-050fc3b495dc None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.476522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2b614-1903-45f9-a4e9-050fc3b495dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:20.478056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b2b614-1903-45f9-a4e9-050fc3b495dc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:20.505411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:20.645056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f7ce44-e901-44d2-acf0-64982d27d723 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:20.677863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 73/143] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:20 2022] POST /identity/v3/users => generated 283 bytes in 519 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:20.679080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2bb7-06dd-4bf8-ad38-689ff18d22fa None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.679366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2bb7-06dd-4bf8-ad38-689ff18d22fa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.679643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5c2bb7-06dd-4bf8-ad38-689ff18d22fa None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.685567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 74/144] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:20.691371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.691703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.693100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.719430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:20.720112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:20.720409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:20.723307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:20.741860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:20.743141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:20.773844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:20.781046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:20.781966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:20.782727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 71/145] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:20.790369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:20.981142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 75/146] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:21.016615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.028134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff03cc9-99bc-4645-bf2b-5981a8a8c78f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.071228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.071979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.072201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.072480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.073218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:21.073525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.074940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.087234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 72/147] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:20 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:21.093134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.224678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.232136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e67b773-5514-4bcc-9223-ff0ec9da1f6f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.278077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.278742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.279023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.279297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] PATH_INFO: `/v3/users/3468051f65e64a1295f6246a758b8de7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.283263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] RBAC: Authorizing `identity:get_user(user_id=3468051f65e64a1295f6246a758b8de7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.284863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.292492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 76/148] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/users/3468051f65e64a1295f6246a758b8de7 => generated 289 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:21.293779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.423876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c43e1-94ef-42a8-9317-8669b5c3e64f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.464583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.465266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.465548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.465819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.466963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.468439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.470148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 73/149] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:21.476912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.619836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef74527c-e9e9-4b79-87e7-b98390757db4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.650928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.705457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e633084-00f2-4bc1-ae71-aee1acf25f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.706174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e633084-00f2-4bc1-ae71-aee1acf25f5b None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.706459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e633084-00f2-4bc1-ae71-aee1acf25f5b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.706737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e633084-00f2-4bc1-ae71-aee1acf25f5b None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/3468051f65e64a1295f6246a758b8de7/roles/233bfd8b4640483288d9822efc76cb09` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.708754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.715678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e633084-00f2-4bc1-ae71-aee1acf25f5b None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bd3cad3b8f49989950c0a9e6f1976c, user_id=3468051f65e64a1295f6246a758b8de7, role_id=233bfd8b4640483288d9822efc76cb09)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:21.717214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e633084-00f2-4bc1-ae71-aee1acf25f5b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:21.810901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db899b2-993a-40c7-a4e9-e71c17e112db None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:21.846555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 77/150] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:21 2022] PUT /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/users/3468051f65e64a1295f6246a758b8de7/roles/233bfd8b4640483288d9822efc76cb09 => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:21.850237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01385887-d31f-4586-845f-ce6850e40c25 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.850237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01385887-d31f-4586-845f-ce6850e40c25 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.850237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01385887-d31f-4586-845f-ce6850e40c25 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.851286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 78/151] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:21.858546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.858876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.859109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.863194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:21.865208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:21.865884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:21.866432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:21.882480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:21.884216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:21.892430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:21.899585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:21.900527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:21.901341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:21.903277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 74/152] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:21.909360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:21.916256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:21.991403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 79/153] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:22.077178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.104998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef6983c-beb6-42e4-9743-081579c107e3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.151895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.152514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.152791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.153086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.153766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:22.154083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.155447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.170541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 75/154] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:22.183644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.396125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d9cbf6-3087-44c3-8100-481d26c00ef1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.464276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.465844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.466595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.467358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.469751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.472809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.477226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 80/155] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:22.482793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.615984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:22.675162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b525ab9-9e9c-4e3a-be73-a8559b810ec7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.705379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.778545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.779177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.779451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.779723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:22.806148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:22.895358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:22.897515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:22.907566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c7657-a236-4de1-b122-bcb358ec8ec6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:22.994428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883af8f9-d4fd-4308-86b9-0957d41bc573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:22.995438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af8f9-d4fd-4308-86b9-0957d41bc573 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:22.995877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af8f9-d4fd-4308-86b9-0957d41bc573 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:22.996423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883af8f9-d4fd-4308-86b9-0957d41bc573 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.024768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883af8f9-d4fd-4308-86b9-0957d41bc573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-883af8f9-d4fd-4308-86b9-0957d41bc573 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:23.044036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:23.044961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:23.045752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:23.047121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 81/156] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 432 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:23.048491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6d01-5143-4e0c-b351-b75eae7c6ccd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.048982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6d01-5143-4e0c-b351-b75eae7c6ccd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.053915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fb6d01-5143-4e0c-b351-b75eae7c6ccd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.056488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 82/157] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:23.057361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 76/158] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 575 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:23.057901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb6d01-5143-4e0c-b351-b75eae7c6ccd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.058393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.131781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.280303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5ded99-e70e-40d1-9ced-6b3c43835606 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.280925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.325240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fb6d01-5143-4e0c-b351-b75eae7c6ccd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.418319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e18a14-ef25-4692-82be-fee84b524eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.419899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18a14-ef25-4692-82be-fee84b524eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.420707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18a14-ef25-4692-82be-fee84b524eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.421444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e18a14-ef25-4692-82be-fee84b524eb6 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.421843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.423331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.423818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.424390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.424785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e18a14-ef25-4692-82be-fee84b524eb6 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.426533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e18a14-ef25-4692-82be-fee84b524eb6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.429377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 77/159] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:23.434082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:23.434082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.434997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.437118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.437860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.438557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.462054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 83/160] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:23.463682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.481614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:23.483217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:23.635581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 78/161] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:23.636898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:23.711874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eab215c-01cd-4633-b72d-9c3539d21ca8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.733110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.782487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.783115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.783396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.783671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.784414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=0a614963f72e4c8abe5f162fabb45136) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:23.784719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.787272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea1f05-2800-48d7-a377-9dab76bed72e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.808492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 84/162] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/role_assignments?user.id=0a614963f72e4c8abe5f162fabb45136&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 608 bytes in 346 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:23.814699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcefab-9f60-483c-a206-85e1478b0780 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.815209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcefab-9f60-483c-a206-85e1478b0780 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.815711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefcefab-9f60-483c-a206-85e1478b0780 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.829551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 85/163] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:23 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:23.831025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.831310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.831583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.877478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb754e7-07b1-42be-a64c-7dbb048fbc50 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:23.886326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:23.888195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:23.892209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:23.965917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:23.966595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:23.966876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:23.967147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:23.968456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:23.969986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:23.971865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 79/164] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:23.987271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.134392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 86/165] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:24.248401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe14068-d64e-45e3-a9cb-db38e9492bdd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:24.272554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:24.327220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:24.328252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.328753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.329282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.448172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:24.450292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:24.599904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 80/166] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 613 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:09:24.609236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.609729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.627121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:24.758989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c05e8e-a893-435d-8ae9-476987c7af8c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:24.871790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:24.872902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:24.873399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:24.873838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:24.910182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:24.921329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:24.922275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:24.923325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:24.925755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 81/167] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:24.934436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:24.968314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b42e602-25c8-4c12-a4f7-a27a09704bbb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.055547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.056546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.057056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.057565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] PATH_INFO: `/v3/projects/4aa12113285c469683047af9078a81ba` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.059583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] RBAC: Authorizing `identity:get_project(project_id=4aa12113285c469683047af9078a81ba)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.062282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.069271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 87/168] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/projects/4aa12113285c469683047af9078a81ba => generated 312 bytes in 461 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:25.074424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.078647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.101043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.137297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be11ae-d943-4386-8850-54f6e5dc16a3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.221187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.222207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.222686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.223178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.224890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:25.225447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.227146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.245664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 82/169] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:24 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:25.253076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.348131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0caf55f-f032-4ae1-b55c-ff12a516c925 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.476921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.477938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.478394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.478838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.480106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=4aa12113285c469683047af9078a81ba, user.id=0a614963f72e4c8abe5f162fabb45136) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:25.480615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.483365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.505113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 88/170] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/role_assignments?user.id=0a614963f72e4c8abe5f162fabb45136&scope.project.id=4aa12113285c469683047af9078a81ba&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 608 bytes in 427 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:25.506389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.591491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3d26dc-8ce0-4cb7-ba0c-34ba72674bfb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.693800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.702833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.704109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.704109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.704375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] PATH_INFO: `/v3/users/0a614963f72e4c8abe5f162fabb45136` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.706613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] RBAC: Authorizing `identity:get_user(user_id=0a614963f72e4c8abe5f162fabb45136)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:25.708749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:25.711528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 83/171] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/users/0a614963f72e4c8abe5f162fabb45136 => generated 275 bytes in 459 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:25.735709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:25.768529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:25.873528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6061280c-4bec-4030-8bd5-01d2a0ee9c6d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.954975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:25.955641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:25.955920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:25.956190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:25.959796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06436845-09fe-4262-b50b-23e982739aa6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:25.991114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:25.997416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:25.999155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:26.000843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 89/172] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 493 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:26.015277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.077468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2059c4cb-daa9-40b2-9e5f-648999c1df37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.078101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c4cb-daa9-40b2-9e5f-648999c1df37 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.078483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c4cb-daa9-40b2-9e5f-648999c1df37 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.078645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2059c4cb-daa9-40b2-9e5f-648999c1df37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.231006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e19a1e3-b353-444e-84c7-7a90662e7e14 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.237515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.245834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c4cb-daa9-40b2-9e5f-648999c1df37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.247370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2059c4cb-daa9-40b2-9e5f-648999c1df37 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.299067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.307880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.309546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.309827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.310102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.311609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:26.311916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.313368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31597aa2-7828-4984-9340-5a13979203a6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.338041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 90/173] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:26.341298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff35f73-3be7-480a-8716-e542a147d832 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.345314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff35f73-3be7-480a-8716-e542a147d832 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.345314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff35f73-3be7-480a-8716-e542a147d832 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.347490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 91/174] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:26 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:26.349036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff35f73-3be7-480a-8716-e542a147d832 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.444150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 84/175] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 709 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:09:26.445311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b3a3-b66d-48bb-a7cd-24b0dfc17612 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.445679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b3a3-b66d-48bb-a7cd-24b0dfc17612 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.445966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8b3a3-b66d-48bb-a7cd-24b0dfc17612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.467909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dda8b3a3-b66d-48bb-a7cd-24b0dfc17612 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:26.469153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dda8b3a3-b66d-48bb-a7cd-24b0dfc17612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:26.559274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 85/176] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:26.560669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8565f0-5f66-4503-8f66-76c3c28596cd None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.560948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8565f0-5f66-4503-8f66-76c3c28596cd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.561250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8565f0-5f66-4503-8f66-76c3c28596cd None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.569460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 86/177] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:26.569460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8565f0-5f66-4503-8f66-76c3c28596cd None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.580605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff35f73-3be7-480a-8716-e542a147d832 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.605780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.674109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ac5a11-c45c-48d0-8016-85eb4d7b1334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.675394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5a11-c45c-48d0-8016-85eb4d7b1334 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.677411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5a11-c45c-48d0-8016-85eb4d7b1334 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.678135 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ac5a11-c45c-48d0-8016-85eb4d7b1334 None admin] PATH_INFO: `/v3/users/0f6d02bd20804957aa92efbfa08393a0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.680370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac5a11-c45c-48d0-8016-85eb4d7b1334 None admin] RBAC: Authorizing `identity:get_user(user_id=0f6d02bd20804957aa92efbfa08393a0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.682607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ac5a11-c45c-48d0-8016-85eb4d7b1334 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.689264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 92/178] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/users/0f6d02bd20804957aa92efbfa08393a0 => generated 247 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:26.689264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a96584-dfde-411f-836e-c556ca8627f4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.689264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a96584-dfde-411f-836e-c556ca8627f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.689264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a96584-dfde-411f-836e-c556ca8627f4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.689264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 93/179] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:26.698956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.700108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.701241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.710784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8565f0-5f66-4503-8f66-76c3c28596cd None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.735990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:26.737526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:26.787247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2834dd9f-fc4b-442a-a9f1-6afe1e491eb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:26.789232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834dd9f-fc4b-442a-a9f1-6afe1e491eb2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.789920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834dd9f-fc4b-442a-a9f1-6afe1e491eb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.790599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2834dd9f-fc4b-442a-a9f1-6afe1e491eb2 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.792799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834dd9f-fc4b-442a-a9f1-6afe1e491eb2 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:26.796279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2834dd9f-fc4b-442a-a9f1-6afe1e491eb2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:26.799136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 87/180] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:26.803774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:26.804595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:26.805968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:26.824966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:26.845048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 94/181] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:26.845048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:26.856516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:26.859043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:26.978563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b4666-f431-49c8-906f-756c37742251 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:26.992984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 88/182] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:26.995041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.029292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1681da-552d-44f1-9db7-5187091ceb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.030192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1681da-552d-44f1-9db7-5187091ceb8b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.030375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1681da-552d-44f1-9db7-5187091ceb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.030632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1681da-552d-44f1-9db7-5187091ceb8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.056078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.130373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.148479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1681da-552d-44f1-9db7-5187091ceb8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.150031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1681da-552d-44f1-9db7-5187091ceb8b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.206006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4832d7d9-ace1-44ae-bde3-43b2b25f899e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.291595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.292213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.292494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.292770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.293558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] RBAC: Adding query filter params (role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=0a614963f72e4c8abe5f162fabb45136) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:27.293871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.295345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.302880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 89/183] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/role_assignments?user.id=0a614963f72e4c8abe5f162fabb45136&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 608 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:27.305250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.325122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 95/184] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 480 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:27.325122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51981-90d6-472a-b22b-d7365d1c0100 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.325122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51981-90d6-472a-b22b-d7365d1c0100 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.325122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c51981-90d6-472a-b22b-d7365d1c0100 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.325122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 96/185] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:27.325122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51981-90d6-472a-b22b-d7365d1c0100 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.448323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.455526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.463458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c51981-90d6-472a-b22b-d7365d1c0100 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.470587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f144615-8ffd-4541-afb2-0534fbe8ede7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.517862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.519064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.519555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.519663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.520818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.520978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.521300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.522036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.523238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.524763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.526496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 97/186] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:27.527297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.544544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:27.555244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:27.556812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:27.559277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 90/187] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:27.561723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.689060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5b40cc-7b6b-4091-b727-8a5ea3f7656e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.717088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d8eb0-715c-4942-8b84-2166f234a8b6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:27.726573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34839188-c6d8-44e6-b659-e59539ccbd8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.727222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34839188-c6d8-44e6-b659-e59539ccbd8b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.727497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34839188-c6d8-44e6-b659-e59539ccbd8b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.727764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34839188-c6d8-44e6-b659-e59539ccbd8b None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.728984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34839188-c6d8-44e6-b659-e59539ccbd8b None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:27.730494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34839188-c6d8-44e6-b659-e59539ccbd8b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:27.732273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 98/188] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:27.738559 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.739376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.739648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.759417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:27.760629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:27.772028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:27.772675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:27.772951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:27.773252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:27.786466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.788224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:27.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:27.794915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 91/189] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:27.796567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.819955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:27.864346 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 99/190] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:27.865294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:27.975124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d114d87-bd93-4ec1-8827-1b8ba5fceb82 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.025592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.026492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.026967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.027427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.028646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.031168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.042689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6530b-cb39-4a11-9f44-26234d35a69e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.089759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.090582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.091923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.091923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.092553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:28.092971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.094622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.106504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 100/191] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:28.107395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.132127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:28.168446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 92/192] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:27 2022] POST /identity/v3/users => generated 297 bytes in 373 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:28.169433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.254756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25bb766-1a50-4da9-a458-97a54e378e2b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.290762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:28.306516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.307565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.308218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.308512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.309629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=0f6d02bd20804957aa92efbfa08393a0) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:28.318926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.318926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.334697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e050aa3c-68d3-41c4-a791-1bc262213628 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.353754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 101/193] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/role_assignments?user.id=0f6d02bd20804957aa92efbfa08393a0&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:28.354520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.389116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.390590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.390969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.391378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.392160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:28.392463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.395995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.411587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 93/194] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:28.419970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.536480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:28.585691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fbd453-1786-4040-8a6d-448a89d87415 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.594716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619117b7-d3ef-4f1e-a552-cae74ca52d22 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.649731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:28.655819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.656482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.656769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.657084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] PATH_INFO: `/v3/users/3468051f65e64a1295f6246a758b8de7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.658931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] RBAC: Authorizing `identity:get_user(user_id=3468051f65e64a1295f6246a758b8de7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.662356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.662769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.663979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.664631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.665227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 102/195] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/users/3468051f65e64a1295f6246a758b8de7 => generated 289 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:28.665404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.666642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.691506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:28.702703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:28.704640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 94/196] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:28.706304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.857687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94ee641-560f-4473-9203-4b42e04f2b50 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.879028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de32945e-877d-4e59-bfd5-3a0545930a87 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:28.920073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.920882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.921304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.921664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] PATH_INFO: `/v3/users/71cbd34469e7437ebef575b930797632` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.922937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:28.924002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:28.924634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:28.925163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:28.939334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] RBAC: Authorizing `identity:get_user(user_id=71cbd34469e7437ebef575b930797632)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:28.941122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:28.944338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 103/197] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/users/71cbd34469e7437ebef575b930797632 => generated 283 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:28.946224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:28.960408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.052427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.063088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9034a68f-45c2-4edb-be39-83403680aca8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.074287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.077112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.105789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.106429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.106712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.106982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.107669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:29.108049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.109471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.120804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 104/198] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:29.121788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.235126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.253560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 95/199] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:29.254106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.299325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a724b68-ba6b-47b9-a45c-ac8574c1491b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.359407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.360906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.361722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.362476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.372669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.384464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b422ee-a285-4864-88ec-bc9039eb5d1c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.427119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.427971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.428350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.428630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] PATH_INFO: `/v3/users/71cbd34469e7437ebef575b930797632` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.429906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] RBAC: Authorizing `identity:get_user(user_id=71cbd34469e7437ebef575b930797632)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.431483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.433604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 96/200] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/users/71cbd34469e7437ebef575b930797632 => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:29.434537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.447821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.449358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.529437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.551770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec6ee2f-27e1-4324-8bea-d0bde025eda5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.566655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 105/201] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:29.567958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.596603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.598147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.598787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.599279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.600751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.602660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.604783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 97/202] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:29.606052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.638272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.687181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12df7c7-2187-470b-bb64-986dc4d26475 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.726653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.727337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.727677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.727882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.732382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9239b1fd-a7be-4471-94c8-426fb92ce6fe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.774203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.777362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.778359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.778902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.779397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.780891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.782914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.787904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 98/203] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:29.787904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.810056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.811813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.904876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:29.913520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c2e3c7-25cf-4e9d-a1f3-f1d0dd5edeb0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:29.939149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 106/204] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:29.944819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:29.970117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:29.971311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:29.972165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:29.972957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:29.974005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] RBAC: Adding query filter params (role.id=233bfd8b4640483288d9822efc76cb09, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c, user.id=3468051f65e64a1295f6246a758b8de7) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:29.974553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:29.976918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:29.984781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 99/205] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/role_assignments?user.id=3468051f65e64a1295f6246a758b8de7&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=233bfd8b4640483288d9822efc76cb09 => generated 608 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:29.986052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:30.058902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:30.217839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359681bb-16a4-43cb-81ae-a4b3127b6373 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:30.228675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c95bab-9595-4be9-a637-7418f9ec469b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:30.277483 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:30.279005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:30.279005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:30.279005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:30.279615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=71cbd34469e7437ebef575b930797632) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:30.279935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:30.281659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:30.289872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 107/206] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/role_assignments?user.id=71cbd34469e7437ebef575b930797632&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:30.309425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b930fccf-7b3f-4244-ba27-4fc80da0583b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:30.311138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930fccf-7b3f-4244-ba27-4fc80da0583b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:30.312032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930fccf-7b3f-4244-ba27-4fc80da0583b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:30.312523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b930fccf-7b3f-4244-ba27-4fc80da0583b None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:30.314889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930fccf-7b3f-4244-ba27-4fc80da0583b None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:30.318491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b930fccf-7b3f-4244-ba27-4fc80da0583b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:30.322474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 100/207] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:29 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:30.331578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:30.566517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c10ad-48dd-4a7d-b5a3-0740933d2d0d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:30.636117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:30.637198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:30.637685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:30.638144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:30.639544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] RBAC: Adding query filter params (role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=71cbd34469e7437ebef575b930797632) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:30.640087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:30.642972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f237ff22-bfc4-4330-934d-4c01ba0340a9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:30.647214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:30.661502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 108/208] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:30 2022] GET /identity/v3/role_assignments?user.id=71cbd34469e7437ebef575b930797632&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 259 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:31.132372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68c29b-d27c-4c6c-b29d-11c073975e26 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:31.133404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68c29b-d27c-4c6c-b29d-11c073975e26 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:31.133889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea68c29b-d27c-4c6c-b29d-11c073975e26 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:31.135516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 101/209] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:31.142001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44846ae8-f048-46c6-9d8a-ee17999576b3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:31.142674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44846ae8-f048-46c6-9d8a-ee17999576b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:31.143199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44846ae8-f048-46c6-9d8a-ee17999576b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:31.171957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-44846ae8-f048-46c6-9d8a-ee17999576b3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:31.174006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44846ae8-f048-46c6-9d8a-ee17999576b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:31.308488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 109/210] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:31.781404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68c29b-d27c-4c6c-b29d-11c073975e26 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:31.813541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:31.956233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdf6c3-da8f-4fbe-be0e-0e10c85c46ea None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:31.957003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdf6c3-da8f-4fbe-be0e-0e10c85c46ea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:31.957003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1fdf6c3-da8f-4fbe-be0e-0e10c85c46ea None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:31.960223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 110/211] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:31.964199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea68c29b-d27c-4c6c-b29d-11c073975e26 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:31.969863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:31.970116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:31.970584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.003318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:32.007024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:32.033235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:32.034744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.035457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.036293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.039037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:32.042328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:32.046137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 102/212] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:32.056072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:32.169527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 111/213] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:32.225084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:32.237712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3070c3-04b9-42f3-9918-dc07ee947fa4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:32.305861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75daf54f-06d0-4294-932a-ecd268ba507e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:32.307194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daf54f-06d0-4294-932a-ecd268ba507e None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.307876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daf54f-06d0-4294-932a-ecd268ba507e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.308583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75daf54f-06d0-4294-932a-ecd268ba507e None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.310002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daf54f-06d0-4294-932a-ecd268ba507e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:32.312977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75daf54f-06d0-4294-932a-ecd268ba507e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:32.482270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 103/214] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:32 2022] POST /identity/v3/users => generated 297 bytes in 431 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:32.716642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:32.781520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41a34a-9aff-4639-8992-f1ae873e37f7 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.781924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41a34a-9aff-4639-8992-f1ae873e37f7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.782399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de41a34a-9aff-4639-8992-f1ae873e37f7 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.785099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 104/215] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:32.796330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442853-749e-4599-8460-8396786642db None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.796747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442853-749e-4599-8460-8396786642db None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.797205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55442853-749e-4599-8460-8396786642db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.836204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-55442853-749e-4599-8460-8396786642db None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:32.839624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-55442853-749e-4599-8460-8396786642db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:32.842200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:32.895725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3173b39-b5ba-49db-b2d5-41d9fa457a65 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:32.936517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:32.937208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:32.937480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:32.937814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:32.951529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:32.956960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:32.959958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:32.962695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 112/216] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:32.965264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:32.989818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 105/217] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:33.222486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d697100-22ef-4628-bc3f-45a7c4952b0d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.268025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.310145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.310783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.311066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.311341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.312113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:33.312418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.313880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.314157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55442853-749e-4599-8460-8396786642db None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.343778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 113/218] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:33.351296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.494492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf18049-b06e-4970-8698-2056fcf227ec None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.533602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.534423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.534698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.534926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] PATH_INFO: `/v3/users/059865d31c1147e8beb2f01a5c4e0a6e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.550902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] RBAC: Authorizing `identity:get_user(user_id=059865d31c1147e8beb2f01a5c4e0a6e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.556250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.558062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 114/219] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/users/059865d31c1147e8beb2f01a5c4e0a6e => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:33.561385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.569313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.628432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.652687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55442853-749e-4599-8460-8396786642db None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.692410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6d2bbb-67c0-46dd-b0bd-6ae1eee0631d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.734360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53deddf7-ee50-4b6e-bc65-1715b2db7db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.735416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deddf7-ee50-4b6e-bc65-1715b2db7db0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.735907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deddf7-ee50-4b6e-bc65-1715b2db7db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.736481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deddf7-ee50-4b6e-bc65-1715b2db7db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.778477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8b57d3-2200-4a1b-87e6-7f50d02ed853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.779085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b57d3-2200-4a1b-87e6-7f50d02ed853 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.779364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b57d3-2200-4a1b-87e6-7f50d02ed853 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.779636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8b57d3-2200-4a1b-87e6-7f50d02ed853 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.792566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8b57d3-2200-4a1b-87e6-7f50d02ed853 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0e8b57d3-2200-4a1b-87e6-7f50d02ed853 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:33.797510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:33.798460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:33.799260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 106/220] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 486 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:33.799922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edc515-c08b-41e9-a4d0-4bbd4b631fd1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.805300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edc515-c08b-41e9-a4d0-4bbd4b631fd1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.805300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42edc515-c08b-41e9-a4d0-4bbd4b631fd1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.805300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 107/221] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:33.808925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edc515-c08b-41e9-a4d0-4bbd4b631fd1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:33.853774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53deddf7-ee50-4b6e-bc65-1715b2db7db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.855437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53deddf7-ee50-4b6e-bc65-1715b2db7db0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:33.921739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:33.949476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42edc515-c08b-41e9-a4d0-4bbd4b631fd1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:33.980988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 115/222] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:33.982582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.982879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.985985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.987958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:33.989930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:33.989930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:33.990224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:33.990872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:33.991176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:33.992595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.002946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 108/223] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.003830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.007822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.008533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:34.010401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:34.115548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 116/224] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:34.116922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.146152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1de087-a99d-4ea4-b5fe-90747d12deb5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.211345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.238580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.240064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.240237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5478a7-f824-4d05-ace5-387fe7c59c92 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.240429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.240703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.242005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.243544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.252613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 109/225] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.262277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.289159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.289822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.290104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.290374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] PATH_INFO: `/v3/users/71cbd34469e7437ebef575b930797632` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.293762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] RBAC: Authorizing `identity:get_user(user_id=71cbd34469e7437ebef575b930797632)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.295240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.302486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 117/226] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/users/71cbd34469e7437ebef575b930797632 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.306688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.476598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.477565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dd134-792c-457b-a59c-a46524b8b6bb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.523690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.524623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.525099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.525528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.527058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.529239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.531633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 118/227] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.532944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.545930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.568649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30bf7ac-7fd3-4fab-a75a-030004fb1729 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.623098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.624799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.625325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.625805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.627192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=059865d31c1147e8beb2f01a5c4e0a6e) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:34.627741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.630953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.639301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 110/228] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/role_assignments?user.id=059865d31c1147e8beb2f01a5c4e0a6e&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 259 bytes in 377 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:34.639963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.752364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa887d1-9cf2-4e29-81a0-584addfba4c8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.808966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74397a5-42b7-4a92-afde-cf1c55364d2a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:34.814310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.815359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.815676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.816074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.847755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:34.858103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:34.859084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:34.859863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:34.860468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 119/229] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:34.871528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:34.908569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:34.910031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:34.911765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:34.912448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/71cbd34469e7437ebef575b930797632/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:34.916360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] RBAC: Authorizing `identity:create_grant(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user_id=71cbd34469e7437ebef575b930797632, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:34.919742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:34.925337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:34.980259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:35.025286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b8c5ef-cde3-4df5-bc72-b2876cfa8696 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:35.027933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 111/230] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:34 2022] PUT /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/71cbd34469e7437ebef575b930797632/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:35.078796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:35.079433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.079715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.079988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.080673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:35.080976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:35.082743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:35.092332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 120/231] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:34 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:35.101991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:35.305985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2befcb10-fb9d-4a88-ad85-758b068bd908 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:35.385824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45368442-e364-42bc-b45e-dc384f796778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:35.387221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45368442-e364-42bc-b45e-dc384f796778 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.387877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45368442-e364-42bc-b45e-dc384f796778 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.388532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45368442-e364-42bc-b45e-dc384f796778 None admin] PATH_INFO: `/v3/users/71cbd34469e7437ebef575b930797632` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.390907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45368442-e364-42bc-b45e-dc384f796778 None admin] RBAC: Authorizing `identity:get_user(user_id=71cbd34469e7437ebef575b930797632)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:35.393984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45368442-e364-42bc-b45e-dc384f796778 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:35.397784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 112/232] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:35 2022] GET /identity/v3/users/71cbd34469e7437ebef575b930797632 => generated 283 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:35.406137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:35.507005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58283c14-2ebc-47bb-ad5c-8005c0480433 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.507520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58283c14-2ebc-47bb-ad5c-8005c0480433 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.507787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58283c14-2ebc-47bb-ad5c-8005c0480433 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.517029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 113/233] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:35.519375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b91559-1750-4845-9fbb-8336747df82b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.519727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b91559-1750-4845-9fbb-8336747df82b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.520816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b91559-1750-4845-9fbb-8336747df82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.543920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951a5ca-d649-4afd-b555-8d55ba6d13de None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:35.577640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:35.581967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f0b91559-1750-4845-9fbb-8336747df82b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:35.589036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0b91559-1750-4845-9fbb-8336747df82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:35.589324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:35.589977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.590259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.590536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.592808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:35.594330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:35.603256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 121/234] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:35 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:35.604057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:35.752600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb672d3-1420-4fd5-9696-7d5d94276182 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:35.798298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caee409c-ce25-47af-a47e-28953187228c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:35.799051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee409c-ce25-47af-a47e-28953187228c None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.799336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee409c-ce25-47af-a47e-28953187228c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.799618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caee409c-ce25-47af-a47e-28953187228c None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/71cbd34469e7437ebef575b930797632/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.802664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee409c-ce25-47af-a47e-28953187228c None admin] RBAC: Authorizing `identity:create_grant(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user_id=71cbd34469e7437ebef575b930797632, role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:35.804191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:35.805952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caee409c-ce25-47af-a47e-28953187228c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:35.829522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 114/235] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:35.898774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 122/236] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:35 2022] PUT /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/71cbd34469e7437ebef575b930797632/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:35.954487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7a3bf-fc3f-4ec5-8321-599c3958d58b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.955122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7a3bf-fc3f-4ec5-8321-599c3958d58b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.955679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe7a3bf-fc3f-4ec5-8321-599c3958d58b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:35.957994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 115/237] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:35.975213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:35.975563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:35.975889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:36.002290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:36.003566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:36.079050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:36.113832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 123/238] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:36.239959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7a3bf-fc3f-4ec5-8321-599c3958d58b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:36.399611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe7a3bf-fc3f-4ec5-8321-599c3958d58b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:36.416278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:36.452351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:36.453072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:36.453357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:36.453674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:36.454887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:36.457828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:36.463227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 116/239] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:36.470418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:36.553401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026bb4ad-ee84-42a1-a72a-a1cc7b6becc3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:36.592602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:36.593281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:36.593571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:36.593850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:36.606895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:36.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:36.613112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:36.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 124/240] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:36.622719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:36.634273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:36.696767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:36.720310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c1fbb-cf45-4288-9a09-c81bca6b0044 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:36.769034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:36.772710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:36.773493 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:36.776553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:36.777866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:36.781252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:36.834326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80043d5-1040-4d17-8854-5520968b03b9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:36.886949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 117/241] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:36 2022] POST /identity/v3/users => generated 293 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:36.891634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:36.892575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:36.893005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:36.893541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:36.894462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:36.895555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:36.897502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:36.907881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 125/242] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:36.915818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:37.135842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ba6ebe-0b1d-4a0b-b8e8-9adbad239975 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:37.194458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:37.234252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df47079e-b62c-4e77-a820-dc770fb63471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:37.235896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47079e-b62c-4e77-a820-dc770fb63471 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.236652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47079e-b62c-4e77-a820-dc770fb63471 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.237502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df47079e-b62c-4e77-a820-dc770fb63471 None admin] PATH_INFO: `/v3/users/2f6db1b72da24f8391b936a3d4217bb8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.311987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47079e-b62c-4e77-a820-dc770fb63471 None admin] RBAC: Authorizing `identity:get_user(user_id=2f6db1b72da24f8391b936a3d4217bb8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:37.315643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df47079e-b62c-4e77-a820-dc770fb63471 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:37.325679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 118/243] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:36 2022] GET /identity/v3/users/2f6db1b72da24f8391b936a3d4217bb8 => generated 297 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:37.337544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:37.405859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd1c1-2030-4a50-88cd-ad2e134a934a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.406136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd1c1-2030-4a50-88cd-ad2e134a934a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.406401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45cd1c1-2030-4a50-88cd-ad2e134a934a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.408027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 119/244] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:37.415525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c5852-13d0-46d0-93d4-971d45d33b70 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.415868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c5852-13d0-46d0-93d4-971d45d33b70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.416082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3c5852-13d0-46d0-93d4-971d45d33b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.437801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2f3c5852-13d0-46d0-93d4-971d45d33b70 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:37.439051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2f3c5852-13d0-46d0-93d4-971d45d33b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:37.544203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa561ac-ec28-48c0-9ea5-28a5174b8866 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:37.559167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:37.579822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 120/245] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:37.618462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:37.619588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.620047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.620590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.762638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:37.765538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:37.799789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7597f-d384-433a-a39c-4a00c2027f43 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.800811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7597f-d384-433a-a39c-4a00c2027f43 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.801878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe7597f-d384-433a-a39c-4a00c2027f43 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.807492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 121/246] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:37.811122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:37.811808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:37.812289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:37.839668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:37.841271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:37.939090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 122/247] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:37.940364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:37.972330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 126/248] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:37.978815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:37.993444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.011266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.136579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9f7a83-f020-45bf-ac6b-bab9c41891ff None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.203122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1362821-5629-4e3f-be4a-2d959cdf427f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.212864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.213633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.213914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.214304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.246221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:38.255159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:38.256211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:38.256989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 123/249] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:38.267559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.305223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.305887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.306161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.306433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.307922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:38.310780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:38.314433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 127/250] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:37 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:38.315196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.473354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.513143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.532424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b793d0bd-1744-4e25-b818-f994b06a16a3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.550320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bdba5-36c1-46dd-8d62-8592ca796f94 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.632000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.633083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.633540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.634015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.642667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.643686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.644170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.644705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.646432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:38.646948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:38.653463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:38.670592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:38.672767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 124/251] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:38.675626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:38.682874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:38.683828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:38.684615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:38.687203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 128/252] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 372 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:38.688295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.848875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008b1ad4-1d9f-413d-928f-5e4febb52dfc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.908760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5c04f5-2fa5-49b3-8d03-5fc89e5eaef7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:38.914565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.915206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.915490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.915766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.916543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] RBAC: Adding query filter params (role.id=233bfd8b4640483288d9822efc76cb09, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=2f6db1b72da24f8391b936a3d4217bb8) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:38.916854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:38.918429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:38.929834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 125/253] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/role_assignments?user.id=2f6db1b72da24f8391b936a3d4217bb8&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=233bfd8b4640483288d9822efc76cb09 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:38.935861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:38.985331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:38.994218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3393545-e8ef-4535-8352-3e24f4641c39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:38.994883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3393545-e8ef-4535-8352-3e24f4641c39 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:38.995163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3393545-e8ef-4535-8352-3e24f4641c39 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:38.995440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3393545-e8ef-4535-8352-3e24f4641c39 None admin] PATH_INFO: `/v3/users/059865d31c1147e8beb2f01a5c4e0a6e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:38.996843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3393545-e8ef-4535-8352-3e24f4641c39 None admin] RBAC: Authorizing `identity:get_user(user_id=059865d31c1147e8beb2f01a5c4e0a6e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:38.998378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3393545-e8ef-4535-8352-3e24f4641c39 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.009400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 129/254] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/users/059865d31c1147e8beb2f01a5c4e0a6e => generated 297 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:39.010854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8699b-4616-4989-89f1-76ac8a1c18e2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.011138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8699b-4616-4989-89f1-76ac8a1c18e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.011415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c8699b-4616-4989-89f1-76ac8a1c18e2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.012772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 130/255] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:39.025285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8699b-4616-4989-89f1-76ac8a1c18e2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.031877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.183139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9675c99-ef0a-4639-99c7-8692d36f864a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.227160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.227792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.228075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.228348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.229896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:39.230207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.232388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40583415-2a54-4aa7-9d88-b95d7e397a34 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.240224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c8699b-4616-4989-89f1-76ac8a1c18e2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.242992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 126/256] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:39.247246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.248100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.250664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.278140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:39.279741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:39.331064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.332217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.332712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.333270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.350121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.353166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.356655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 131/257] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:39.362701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.450563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 127/258] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:39.456156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.518049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.615048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:39.691325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a9a96b-9f87-4998-84f6-55cbbfae18b1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.798526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.799160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.799445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.799715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] PATH_INFO: `/v3/users/71cbd34469e7437ebef575b930797632` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.801472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] RBAC: Authorizing `identity:get_user(user_id=71cbd34469e7437ebef575b930797632)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.803126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:39.805093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 132/259] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/users/71cbd34469e7437ebef575b930797632 => generated 283 bytes in 448 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:39.844602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:39.856335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67aef5-fa4c-4cfc-86cb-07fca77b25fe None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:39.966240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:39.967388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:39.967881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:39.968376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/059865d31c1147e8beb2f01a5c4e0a6e/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:39.971954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] RBAC: Authorizing `identity:create_grant(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user_id=059865d31c1147e8beb2f01a5c4e0a6e, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:39.976192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.031124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbabae2-fec7-4fe4-b4ff-664274038be6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.080881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.081478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.081763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.082274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.092053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 128/260] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:39 2022] PUT /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/059865d31c1147e8beb2f01a5c4e0a6e/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:40.098336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.203397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.204910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.250778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.337319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 133/261] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 493 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:40.354757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.424379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.478040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.567385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0469230e-3fd7-4b7b-9fe5-d047e9e168ea None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.577707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac60a3b-e207-463d-ac85-687dc45c0527 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.625275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.625837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.626116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.626392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.639086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:40.644022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:40.645796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:40.647301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 129/262] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:40.658852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.662937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.664026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.664534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.665049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.667136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.670004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.681663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 134/263] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:40.691206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.842343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4baebff0-9eb5-4c9f-acba-fa27e36c67f0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.884440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.892928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.900204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.900204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.900204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.900204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:40.900204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.900204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f5e0fa-32f5-4873-9741-c135a0fa2a53 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.903791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0b9631-e763-48d5-a7d6-13012bb9aeee None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:40.914589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:40.918605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 130/264] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:40.920086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a300c7e-d217-449c-8a83-7a20d97a9ac8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.920443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a300c7e-d217-449c-8a83-7a20d97a9ac8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.920786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a300c7e-d217-449c-8a83-7a20d97a9ac8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.922378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 131/265] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:40.923650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0430f-a651-45a7-806e-bf1811b6417a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.928859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0430f-a651-45a7-806e-bf1811b6417a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.929306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be0430f-a651-45a7-806e-bf1811b6417a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.930876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 132/266] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:40.931825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0430f-a651-45a7-806e-bf1811b6417a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:40.959590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:40.960564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.960990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.961674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:40.963604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=71cbd34469e7437ebef575b930797632) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:40.964107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:40.968022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93dbbd8-cd3a-414f-85e9-0ee1c36b08b5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:40.980461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 135/267] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/role_assignments?user.id=71cbd34469e7437ebef575b930797632&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:40.981998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ecae6-879b-4aec-ba4e-afc279d0f012 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:40.982284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ecae6-879b-4aec-ba4e-afc279d0f012 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:40.982557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ecae6-879b-4aec-ba4e-afc279d0f012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.030961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-791ecae6-879b-4aec-ba4e-afc279d0f012 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:41.034060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-791ecae6-879b-4aec-ba4e-afc279d0f012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:41.211960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 136/268] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:41.211960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.211960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.211960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.214240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be0430f-a651-45a7-806e-bf1811b6417a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.267559 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:41.271668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:41.318206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.319306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.319927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.320290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] PATH_INFO: `/v3/users/71cbd34469e7437ebef575b930797632` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.324874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] RBAC: Authorizing `identity:get_user(user_id=71cbd34469e7437ebef575b930797632)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.327720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.340670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 133/269] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:40 2022] GET /identity/v3/users/71cbd34469e7437ebef575b930797632 => generated 283 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:41.343248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.449143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 137/270] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:41.504159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.567840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58015ce6-343b-4bbe-8211-69668f8e5399 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.614277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.615098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.615379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.615650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.641954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.720582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:41.723071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:41.739826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.815879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:41.869221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8213d9a3-6dc9-4d1a-bc37-514c4a39b12c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:41.882009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 134/271] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 534 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:41.884640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:41.919171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:41.919856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:41.920201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:41.920511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:41.936935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:41.941705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:41.942832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:41.944149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 138/272] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:41.949452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.029939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e451dff9-c500-4841-b894-93fc9c38a58d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.071465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.077765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.078434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.078707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.078977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.080186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.081746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.103595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 135/273] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.103595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.103914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8a700a-26c3-4ba2-9799-666a01a495a2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.142145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fff963-0f75-47d3-a663-e107d4957f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.142808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fff963-0f75-47d3-a663-e107d4957f7d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.143088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fff963-0f75-47d3-a663-e107d4957f7d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.143363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fff963-0f75-47d3-a663-e107d4957f7d None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.145218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fff963-0f75-47d3-a663-e107d4957f7d None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.146424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.147225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fff963-0f75-47d3-a663-e107d4957f7d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.153870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 139/274] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:41 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.153870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a3a02-931b-4be7-8ff9-771a361c7a3a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.153870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a3a02-931b-4be7-8ff9-771a361c7a3a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.153870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82a3a02-931b-4be7-8ff9-771a361c7a3a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.153870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 140/275] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:42.153870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a3a02-931b-4be7-8ff9-771a361c7a3a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.260306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c859d64-469d-48ff-8db9-de4832396bf4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.319076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.320089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.320577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.321078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.322337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:42.322810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.325378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b4e732-5911-4bfd-a382-4155b11f4eb3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.336294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82a3a02-931b-4be7-8ff9-771a361c7a3a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.345336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 136/276] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.345336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.345336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.345336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.377886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:42.380478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:42.398829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.399898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.400337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.400771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.402044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.404404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.501047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 137/277] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:42.501047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.516292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 141/278] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:09:42 2022] POST /identity/v3/users => generated 293 bytes in 363 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:42.523826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.577109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.590388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:42.759424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a184d-c942-45ea-a793-c8969a40203f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.774313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bceda1b-7cde-46da-b82d-4000ab2f4be1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:42.804785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.805833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.806286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.806702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.807624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] RBAC: Adding query filter params (role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=71cbd34469e7437ebef575b930797632) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:42.808085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.809755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.816702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 138/279] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/role_assignments?user.id=71cbd34469e7437ebef575b930797632&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 608 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.822504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3c2244-f6dc-4dd5-b282-0112b81d72cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:42.823415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c2244-f6dc-4dd5-b282-0112b81d72cf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.823858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c2244-f6dc-4dd5-b282-0112b81d72cf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.824278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3c2244-f6dc-4dd5-b282-0112b81d72cf None admin] PATH_INFO: `/v3/users/d37f874ab30140a5ba770b78b55d5c47` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.842882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c2244-f6dc-4dd5-b282-0112b81d72cf None admin] RBAC: Authorizing `identity:get_user(user_id=d37f874ab30140a5ba770b78b55d5c47)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:42.844703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3c2244-f6dc-4dd5-b282-0112b81d72cf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:42.846637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 142/280] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/users/d37f874ab30140a5ba770b78b55d5c47 => generated 293 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:42.854334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:42.877148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d148db9-4302-4cd0-8ba6-20971a7f7901 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.877669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d148db9-4302-4cd0-8ba6-20971a7f7901 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.878108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d148db9-4302-4cd0-8ba6-20971a7f7901 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.880833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 143/281] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:42.890456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:42.891411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:42.892157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:42.931574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:42.935053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:42.997365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.021570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd62b699-713e-4095-8157-4d323d204fe9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.073567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 144/282] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:43.074829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.079369 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.097243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.098812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.099102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.099492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.100246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, user.id=d37f874ab30140a5ba770b78b55d5c47, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:43.100552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.102143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.109117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 139/283] 10.222.0.73 () {56 vars in 1308 bytes} [Tue Aug 23 22:09:42 2022] GET /identity/v3/role_assignments?user.id=d37f874ab30140a5ba770b78b55d5c47&scope.system=all&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 226 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:43.263963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94034c4-7faf-4e01-adb8-10ba7a11b6c6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.341575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.343147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.343945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.344693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.368466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:43.376143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:43.377144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:43.377943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:43.381120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 145/284] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:43.389643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.617429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.648795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b9d45-fc08-404d-97ca-44b64f98a878 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.713511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.728420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.733663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.733663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.733663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.733663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:43.733663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:43.733663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:43.756433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 140/285] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 367 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:43.764750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588ca2af-0860-413b-a62f-5c8e29a1a49c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:43.765096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.798779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:43.813461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:43.814210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:43.814494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:43.814773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:43.836509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:43.843347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:43.844297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:43.845159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:43.845760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 146/286] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:43.852610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:43.976920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d814519-fa80-4e8b-993d-3221aa2c9c62 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.025440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.026479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.026966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.027710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] PATH_INFO: `/v3/users/2f6db1b72da24f8391b936a3d4217bb8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.030229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] RBAC: Authorizing `identity:get_user(user_id=2f6db1b72da24f8391b936a3d4217bb8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.032033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.034083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 141/287] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/users/2f6db1b72da24f8391b936a3d4217bb8 => generated 297 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:44.044512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.161572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5ff5d8-45ef-486e-a1e6-976871825a32 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.209632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.235869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.237500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.237772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.238149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.241249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:44.241563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.243014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.267306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 147/288] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 415 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:44.274551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.286568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.299767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014355b6-f8f9-4522-9ea7-c499e97c3a59 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.344376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.345456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.345950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.346433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.348462 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.350199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.352467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 142/289] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:44.362446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.611002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ff5a6-6f66-4cfc-a6c9-4d3422b8dbbb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.684324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db126b75-71c2-4254-99aa-cde0a945532b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.718056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.718704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.718973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.719244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] PATH_INFO: `/v3/users/059865d31c1147e8beb2f01a5c4e0a6e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.721317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] RBAC: Authorizing `identity:get_user(user_id=059865d31c1147e8beb2f01a5c4e0a6e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.723331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.724976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 148/290] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/users/059865d31c1147e8beb2f01a5c4e0a6e => generated 297 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:44.732224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:44.775607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.776347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.776628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.776897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/2f6db1b72da24f8391b936a3d4217bb8/roles/233bfd8b4640483288d9822efc76cb09` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:44.780181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] RBAC: Authorizing `identity:create_grant(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user_id=2f6db1b72da24f8391b936a3d4217bb8, role_id=233bfd8b4640483288d9822efc76cb09)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:44.781677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:44.821473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:44.875014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 143/291] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:09:44 2022] PUT /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/users/2f6db1b72da24f8391b936a3d4217bb8/roles/233bfd8b4640483288d9822efc76cb09 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:44.886323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6dba20-9d6c-4c00-9d91-81fe0d738819 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:44.942268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:44.943001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:44.943363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:44.943727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.057710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.059277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.129890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.195493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 149/292] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:09:45.201582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.227660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.368053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25560d1a-3ec3-4387-becb-1dc410d98326 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.417694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb74d20-ab50-4851-81f9-a98d6872876c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.418262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb74d20-ab50-4851-81f9-a98d6872876c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.418541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb74d20-ab50-4851-81f9-a98d6872876c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.418807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb74d20-ab50-4851-81f9-a98d6872876c None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.419993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb74d20-ab50-4851-81f9-a98d6872876c None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.421476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb74d20-ab50-4851-81f9-a98d6872876c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.423448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 144/293] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.430173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:45.615081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9911dae4-4cce-4ebc-a8c4-8c3f9c3ceb8b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:45.670034 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:45.670574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.670852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.671124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.671860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=059865d31c1147e8beb2f01a5c4e0a6e) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:45.672163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:45.673642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bcdb5c-7781-4f1b-8e96-01ec9af6b836 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:45.682249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 150/294] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:45 2022] GET /identity/v3/role_assignments?user.id=059865d31c1147e8beb2f01a5c4e0a6e&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 608 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:45.811421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9c5de-136b-43df-9c2b-41c152586ac1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.811848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9c5de-136b-43df-9c2b-41c152586ac1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.812286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e9c5de-136b-43df-9c2b-41c152586ac1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.814810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 145/295] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:45.821218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bea2-7c03-43d6-9d35-9906184fe9f7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:45.821504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bea2-7c03-43d6-9d35-9906184fe9f7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:45.822354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0451bea2-7c03-43d6-9d35-9906184fe9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:45.847240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0451bea2-7c03-43d6-9d35-9906184fe9f7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:45.849359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0451bea2-7c03-43d6-9d35-9906184fe9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:45.867032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:45.933606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 151/296] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:46.004866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8f8e-794f-4331-bc20-26a552831bb1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.005398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8f8e-794f-4331-bc20-26a552831bb1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.005873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5edb8f8e-794f-4331-bc20-26a552831bb1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.008553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 146/297] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:46.014857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.015205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.015402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.036543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:46.038220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:46.075080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d765ac-b025-46f7-a74e-a73d1ec85b5c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.075504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d765ac-b025-46f7-a74e-a73d1ec85b5c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.076261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d765ac-b025-46f7-a74e-a73d1ec85b5c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.085169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 147/298] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:46.090988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.091863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.092570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.125713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 152/299] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:46.133321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:46.135401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:46.270381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 148/300] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:46.279323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.365638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:46.388832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2c1a3c-4237-408a-9eeb-b1b8778241ad None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.422125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.427368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.428041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.428318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.428588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.429792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:46.431233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:46.433597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 153/301] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:46.433716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:46.440122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.542761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddb03d3-118f-41b2-8fbf-16e95d7ff8b4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.564725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b4e72d-e0de-4e0f-8d4f-ab262e7634d9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.586574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.587276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.587556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.587826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.602154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:46.607848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:46.608832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:46.609999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:46.617132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 149/302] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:46.617306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.617306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.617306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.617306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] PATH_INFO: `/v3/groups` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.618200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:46.621174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:46.622395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.703675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:46.729046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 154/303] 10.222.0.73 () {60 vars in 1134 bytes} [Tue Aug 23 22:09:46 2022] POST /identity/v3/groups => generated 229 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:09:46.778988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f858b85-40ae-4134-8007-d20a67c6f8a5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:46.825629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:46.833353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:46.854337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:46.855555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:46.860394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:46.860394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:46.860394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:46.860394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:46.861245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:46.878364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 150/304] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:46.888902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.031309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d7e4be-a6d8-44e0-9dc9-318ec9fe21ee None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.074395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.074986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.075277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.075526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.088192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:47.093527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:47.097354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:47.099269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 155/305] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:47.103511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cab1cc-111e-4230-9919-31bdc5c089f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.106340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.153693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.168900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.170384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.174854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.174854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] PATH_INFO: `/v3/users/22872878dd2f4902949b1757b8f14a61` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.189806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.202597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] RBAC: Authorizing `identity:get_user(user_id=22872878dd2f4902949b1757b8f14a61)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.215072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.215072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 151/306] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:46 2022] GET /identity/v3/users/22872878dd2f4902949b1757b8f14a61 => generated 293 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:47.215072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.254411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4e5a9f-6d88-488a-b73d-d740c2f1da49 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.319821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.320445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.320725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.320997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.321747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:47.322706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.324130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.337090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 156/307] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:47.346834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.378058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a20e99-8428-41a6-8318-8afcdc56a4ba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.438042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.439626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.439862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.440497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.442775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] RBAC: Adding query filter params (role.id=233bfd8b4640483288d9822efc76cb09, user.id=22872878dd2f4902949b1757b8f14a61, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:47.443620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.446641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.457563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 152/308] 10.222.0.73 () {56 vars in 1308 bytes} [Tue Aug 23 22:09:47 2022] GET /identity/v3/role_assignments?user.id=22872878dd2f4902949b1757b8f14a61&scope.system=all&role.id=233bfd8b4640483288d9822efc76cb09 => generated 226 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:47.482434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad27f4a8-827d-4f3c-8f8d-63c84f997abc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.515228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.527059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a7cf53-bd61-421c-97ed-b706d65bd6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.527992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7cf53-bd61-421c-97ed-b706d65bd6a1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.532377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7cf53-bd61-421c-97ed-b706d65bd6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.532377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a7cf53-bd61-421c-97ed-b706d65bd6a1 None admin] PATH_INFO: `/v3/users/d37f874ab30140a5ba770b78b55d5c47` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.532377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7cf53-bd61-421c-97ed-b706d65bd6a1 None admin] RBAC: Authorizing `identity:get_user(user_id=d37f874ab30140a5ba770b78b55d5c47)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.532377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a7cf53-bd61-421c-97ed-b706d65bd6a1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.533792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 157/309] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:47 2022] GET /identity/v3/users/d37f874ab30140a5ba770b78b55d5c47 => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:47.541251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:47.631853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:47.689825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645130d1-5a54-480e-b68c-6912dc00d5fb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:47.705672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce038965-0165-4666-8268-e96531f68df2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.706338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce038965-0165-4666-8268-e96531f68df2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.707081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce038965-0165-4666-8268-e96531f68df2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.709960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 158/310] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:47.719128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.719729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.720315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.729494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:47.730169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:47.730455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:47.730738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] PATH_INFO: `/v3/system/users/d37f874ab30140a5ba770b78b55d5c47/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:47.732232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=d37f874ab30140a5ba770b78b55d5c47, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:47.733957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:47.756836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:47.759194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:47.840563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 153/311] 10.222.0.73 () {58 vars in 1334 bytes} [Tue Aug 23 22:09:47 2022] PUT /identity/v3/system/users/d37f874ab30140a5ba770b78b55d5c47/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:47.888941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 159/312] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:48.386950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.567021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e6e074-23ff-406a-a37c-e9941e10c24f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.582648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.621074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.622137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.622614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.623083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.643960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:48.655205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:48.656161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:48.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:48.658082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 154/313] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:48.665676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:48.852927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4ad248-c214-4d72-afb1-5386497b0d7e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:48.898863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:48.918273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:48.919283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:48.919724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:48.920156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:48.921313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:48.921829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:48.924400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:48.945297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 160/314] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:48.947425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.149276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f375326-36b4-4292-8a77-aa31b75f42d1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.217731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107fc93e-f909-4b18-9e50-7f17d5d7b3ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.218835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fc93e-f909-4b18-9e50-7f17d5d7b3ac None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.219344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fc93e-f909-4b18-9e50-7f17d5d7b3ac None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.219849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107fc93e-f909-4b18-9e50-7f17d5d7b3ac None admin] PATH_INFO: `/v3/users/2f6db1b72da24f8391b936a3d4217bb8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.222035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fc93e-f909-4b18-9e50-7f17d5d7b3ac None admin] RBAC: Authorizing `identity:get_user(user_id=2f6db1b72da24f8391b936a3d4217bb8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.225224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107fc93e-f909-4b18-9e50-7f17d5d7b3ac None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.229064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 155/315] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:48 2022] GET /identity/v3/users/2f6db1b72da24f8391b936a3d4217bb8 => generated 297 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:49.234702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.426806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d12912-edb1-4b7d-95f8-85a83923a867 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.438512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314aee8-30a9-4d0c-ba1b-5db43e2a12be None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.439159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314aee8-30a9-4d0c-ba1b-5db43e2a12be None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.439159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2314aee8-30a9-4d0c-ba1b-5db43e2a12be None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.440535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 156/316] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:49.446343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.446573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.446853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.465200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.466169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.466631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.466740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:49.467132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.467967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:49.515834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.532490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.548484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.549730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.583536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 157/317] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:49.677254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 161/318] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:49.680112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.825134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3168b9-dfb6-4884-a80c-68ecfa486448 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:49.858210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.864145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:49.865262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:49.865750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:49.866226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:49.867609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:49.869325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:49.871535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 158/319] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:49.881414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:49.938739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:49.995647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.011845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83dd6ae9-1f74-430c-8947-1869f395e4b1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.023163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d29ce5-26bb-4cbe-9f11-eaee67f28d2f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.058311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.058925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.059199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.059490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.060285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] RBAC: Adding query filter params (role.id=233bfd8b4640483288d9822efc76cb09, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36, user.id=2f6db1b72da24f8391b936a3d4217bb8) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:50.060587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.062062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16cbc077-6d13-4a8f-a20a-f5bfd51d6333 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.069286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 159/320] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/role_assignments?user.id=2f6db1b72da24f8391b936a3d4217bb8&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=233bfd8b4640483288d9822efc76cb09 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:50.070547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914112a0-54b8-458e-907f-5cce89a92cbb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.070815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914112a0-54b8-458e-907f-5cce89a92cbb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.071079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914112a0-54b8-458e-907f-5cce89a92cbb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.073048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 160/321] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:50.081055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f6d4a-82b6-47c4-93d7-80f1d970f3c8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.081683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f6d4a-82b6-47c4-93d7-80f1d970f3c8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.082163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361f6d4a-82b6-47c4-93d7-80f1d970f3c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.093919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.094978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.095387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.095781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.098477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.102210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.103863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-361f6d4a-82b6-47c4-93d7-80f1d970f3c8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:50.105381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-361f6d4a-82b6-47c4-93d7-80f1d970f3c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:50.110488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 162/322] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:09:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:50.114580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.216687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 161/323] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:50.264274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-930ffb43-be76-41a1-b25d-7b74f64e8310 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.290830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde860e1-8c84-48e2-b763-4dacbf7971b6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.291160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde860e1-8c84-48e2-b763-4dacbf7971b6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.291394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde860e1-8c84-48e2-b763-4dacbf7971b6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.292759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 162/324] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:50.298354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.298938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.299468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.319970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.320591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.320868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.321173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] PATH_INFO: `/v3/groups` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.321817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:50.323598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:50.327600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:50.329870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:50.373307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.431071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 163/325] 10.222.0.73 () {60 vars in 1134 bytes} [Tue Aug 23 22:09:50 2022] POST /identity/v3/groups => generated 226 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:09:50.435718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 163/326] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:50.582971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.612207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:50.776261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33e0278-3970-4102-b490-ef0a84a5aae8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:50.781839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.846625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:50.849644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:50.849644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:50.849644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:50.867704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:50.873792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:50.874723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:50.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:50.878049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 164/327] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:50.883913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:50.966991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a901cae-ecb1-4451-9087-8e01b6a1ea43 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.029638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.051706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.052743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.053434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.053899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.073481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:51.078457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:51.080140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:51.081852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:51.083850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607e2046-f280-4d01-8af2-77211b8d33c3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.084086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 164/328] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:09:51.086574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.121734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.149813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.150444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.150789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.151065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.151804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:51.152113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.153951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.167174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 165/329] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:51.176397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.260111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34fdc5-4f09-46ea-b1a0-1792c1b31cf6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.308987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.309691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.309974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.310242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.310927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:51.311227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.312639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.334677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 165/330] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:51.340560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.407204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20c23c4-9f89-4991-9b37-a169950727e7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.482441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.484823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.485586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.486331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] PATH_INFO: `/v3/users/22872878dd2f4902949b1757b8f14a61` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.490919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] RBAC: Authorizing `identity:get_user(user_id=22872878dd2f4902949b1757b8f14a61)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.492787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.496179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 166/331] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/users/22872878dd2f4902949b1757b8f14a61 => generated 293 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:51.507157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.516030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.540349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:51.582437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33770d4-02f5-41f3-a41a-cae0fe45755a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.674721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.677261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.678041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.678756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] PATH_INFO: `/v3/users/d37f874ab30140a5ba770b78b55d5c47` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.681108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] RBAC: Authorizing `identity:get_user(user_id=d37f874ab30140a5ba770b78b55d5c47)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.685235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.688069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 166/332] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/users/d37f874ab30140a5ba770b78b55d5c47 => generated 293 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:51.695327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:51.765419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc77ac4-b271-4fbb-bf2b-4b32150af90e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:51.822343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55f43c5-17b7-41ce-a3f9-ef2e0bfe4001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:51.824245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f43c5-17b7-41ce-a3f9-ef2e0bfe4001 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:51.824750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f43c5-17b7-41ce-a3f9-ef2e0bfe4001 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:51.825249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55f43c5-17b7-41ce-a3f9-ef2e0bfe4001 None admin] PATH_INFO: `/v3/system/users/22872878dd2f4902949b1757b8f14a61/roles/233bfd8b4640483288d9822efc76cb09` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:51.828828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f43c5-17b7-41ce-a3f9-ef2e0bfe4001 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=22872878dd2f4902949b1757b8f14a61, role_id=233bfd8b4640483288d9822efc76cb09)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:51.831825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55f43c5-17b7-41ce-a3f9-ef2e0bfe4001 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:51.921700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 167/333] 10.222.0.73 () {58 vars in 1334 bytes} [Tue Aug 23 22:09:51 2022] PUT /identity/v3/system/users/22872878dd2f4902949b1757b8f14a61/roles/233bfd8b4640483288d9822efc76cb09 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:09:51.935453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdadd69b-a4ab-46f2-b2c6-2e3c1ccd2b85 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:52.004507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:52.005501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:52.005903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:52.006291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:52.007408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] RBAC: Adding query filter params (role.id=87ab2740a8bb4a509daf398f95e522b1, user.id=d37f874ab30140a5ba770b78b55d5c47, scope.system=all) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:52.008001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:52.010281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeee0b79-932e-4dd3-a01d-31f2495ddd38 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:52.032225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 167/334] 10.222.0.73 () {56 vars in 1308 bytes} [Tue Aug 23 22:09:51 2022] GET /identity/v3/role_assignments?user.id=d37f874ab30140a5ba770b78b55d5c47&scope.system=all&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 510 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:53.260662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f1eb-72be-4e3a-b328-70719e5e8293 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.262135 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f1eb-72be-4e3a-b328-70719e5e8293 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.262879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a22f1eb-72be-4e3a-b328-70719e5e8293 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.266162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 168/335] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:53 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:53.272649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:53.274070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:53.274863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:53.312713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:53.315209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:53.334655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:53.461343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 168/336] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:53.971236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f1eb-72be-4e3a-b328-70719e5e8293 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.118657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.194107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a22f1eb-72be-4e3a-b328-70719e5e8293 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.271703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.272760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.273182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.273608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.303971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:54.318767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:54.320522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:54.321975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 169/337] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:54.334746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.582320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6701b611-b96d-44b1-b25a-9df8c75cf260 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.655400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:54.656433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.656923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.657446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.658684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:54.659214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:54.662247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036d57d8-3669-4728-b6f5-3c3ec2fa976e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:54.683331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 169/338] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 350 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:54.694822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:54.752521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7743c6-7f71-4823-ba0a-04eae8bedee9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.753028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7743c6-7f71-4823-ba0a-04eae8bedee9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.753574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7743c6-7f71-4823-ba0a-04eae8bedee9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.756416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 170/339] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:54 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:09:54.766262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:54.766764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:54.767251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:54.787548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.817151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:54.819050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:54.934543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d5ddb0-6ccf-4870-b433-855804623666 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:54.977577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:54.979005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 171/340] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:55.000258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2119ec-e193-483b-bab7-94d17db6f320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.001766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2119ec-e193-483b-bab7-94d17db6f320 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.002518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2119ec-e193-483b-bab7-94d17db6f320 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.003357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2119ec-e193-483b-bab7-94d17db6f320 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.151733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2119ec-e193-483b-bab7-94d17db6f320 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.154736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e2119ec-e193-483b-bab7-94d17db6f320 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.391203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 170/341] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 698 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:55.402010 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.643400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd2b89-abcc-4a89-95da-0e1f8a981249 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:55.668709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087beb07-ecad-43f6-98b4-8fa8700c673a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.668947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087beb07-ecad-43f6-98b4-8fa8700c673a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.669301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087beb07-ecad-43f6-98b4-8fa8700c673a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.670953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 171/342] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:55.676954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.677273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.677581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.722379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:55.724445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:55.749604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:55.750416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:55.750704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:55.750999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:55.753382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:55.755646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:55.758019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 172/343] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:55.758631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:55.768833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:55.771767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:55.832444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 172/344] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:55.833427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.026597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110d0719-3d23-4dbb-a68e-30809429f601 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.041686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf27446-416e-48bf-ad09-7bba77d7d29a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.098319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.099296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.099578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.100480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.105868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483238e-41de-4953-923b-76f389d7505a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.106528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483238e-41de-4953-923b-76f389d7505a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.106813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483238e-41de-4953-923b-76f389d7505a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.107086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d483238e-41de-4953-923b-76f389d7505a None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.121420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483238e-41de-4953-923b-76f389d7505a None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.123019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d483238e-41de-4953-923b-76f389d7505a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.125151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 173/345] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:56.134118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:56.134557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483238e-41de-4953-923b-76f389d7505a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:56.139903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:56.141688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:56.143236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 173/346] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:56.154437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.199348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:56.324278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:56.422059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d483238e-41de-4953-923b-76f389d7505a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.436661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe535a73-bdfc-40bc-af1b-23e4a1a25da4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.526789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.527429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.527713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.528069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.528739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:56.529182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.530587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.542620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.543270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.543567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.543804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.545544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:56.545850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:56.547557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:56.552028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 174/347] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 260 bytes in 418 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:56.552934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.574473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 174/348] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:09:56.582265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:56.760848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:56.792137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafae9b-09e9-4bb3-9abf-ff1dc1d21b32 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.864916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d320e75-5f44-453e-ab77-5fda1b8c7ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:56.865939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d320e75-5f44-453e-ab77-5fda1b8c7ce7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.866429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d320e75-5f44-453e-ab77-5fda1b8c7ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.866913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d320e75-5f44-453e-ab77-5fda1b8c7ce7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.909049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d320e75-5f44-453e-ab77-5fda1b8c7ce7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7d320e75-5f44-453e-ab77-5fda1b8c7ce7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:56.923017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:56.924006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:09:56.924841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:56.927842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 175/349] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:56.929594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b821b1-280e-4be0-9b8e-c7dec0b5817c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.930123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b821b1-280e-4be0-9b8e-c7dec0b5817c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.930600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b821b1-280e-4be0-9b8e-c7dec0b5817c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.932684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 176/350] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:56.936702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbdb3d-48ff-47de-bdeb-bddde0b9d58a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:56.940743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:56.941376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:56.941959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:56.970963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:56.973258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:56.986776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.007547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4415d2-4549-4bb4-8a73-bd189f5abc1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.008504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4415d2-4549-4bb4-8a73-bd189f5abc1f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.008962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4415d2-4549-4bb4-8a73-bd189f5abc1f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.009441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4415d2-4549-4bb4-8a73-bd189f5abc1f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.120926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 177/351] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:09:57.122281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.143026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4415d2-4549-4bb4-8a73-bd189f5abc1f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.145718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4415d2-4549-4bb4-8a73-bd189f5abc1f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.319400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.339602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 175/352] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:57.341382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218962-ac73-4d94-a069-e178d04a83c2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.341902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218962-ac73-4d94-a069-e178d04a83c2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.342379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9218962-ac73-4d94-a069-e178d04a83c2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.343974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 176/353] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:57.346111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108693d5-26bc-4bea-8299-24bad9a81a5f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.346649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108693d5-26bc-4bea-8299-24bad9a81a5f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.347693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108693d5-26bc-4bea-8299-24bad9a81a5f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.349385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 177/354] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:57.354956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6094f12-1db7-4f12-8fa5-5337fbeca58c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.354956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6094f12-1db7-4f12-8fa5-5337fbeca58c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.354956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6094f12-1db7-4f12-8fa5-5337fbeca58c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.382953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478b72f2-9ed4-4850-af44-c39ec4de89e0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.393051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d6094f12-1db7-4f12-8fa5-5337fbeca58c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:57.400233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d6094f12-1db7-4f12-8fa5-5337fbeca58c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:57.434767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.462515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.463611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.464189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.464769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.465967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:57.466489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.468870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.487338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 178/355] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:57.490227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.527499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 178/356] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:57.529101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.529394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.529670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.552606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:09:57.553978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:09:57.633059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 179/357] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:09:57.633915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.737632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.756532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c19df6b-f359-4c87-9464-5897674f08d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.798946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.799478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.799825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.800119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.833351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bcb038-4bc2-43fc-9259-299f6f5e3295 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:57.883898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.886516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.947379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:57.948419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:57.948903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:57.949418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:57.954674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:57.958805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:57.962242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 179/358] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:57 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 474 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:57.967570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:57.988721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:57.993127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.008872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 180/359] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:58.009814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.126837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c982e8-a7d0-4ba7-9d27-72ca5afc473a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.181662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.182763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.183261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.183754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.185109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:58.186619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.188351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 181/360] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:58.189388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.220224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de91f6c-4db6-46e0-8697-a3eed15bdd8e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.250495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.287202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.288250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.289123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.290259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.307960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:58.313307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6e785d-09ec-4292-ad60-589ffaafd22e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:58.314762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:58.315759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:09:58.316558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:09:58.319169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 180/361] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:09:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:58.320261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.375149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.376261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.376761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.381060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.400695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:58.412747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:58.413796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:58.414635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 182/362] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:58.415314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.417108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.467163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68357c9-bc7f-47e6-b270-2ee6e1009450 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.523693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.524649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.524936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.525241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.527003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:58.528826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.531761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 181/363] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:58.532950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.599469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c29441-d30b-4ffd-b255-7dbbc5effb68 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.616759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.660824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.661689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.661974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.662249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.681034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:09:58.690140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:09:58.691942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:09:58.692895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:09:58.697060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 183/364] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:09:58.697060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.707454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3595cd9b-2b3b-454d-989d-0ff15e6d3213 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.760608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.761313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.761599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.761874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.763275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:58.763596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:58.765118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.767469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:58.772645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 182/365] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:58.773800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:58.898618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8385102d-c2fd-4feb-86f7-0ef12a3e147a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.952464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d552d6-923c-4122-b671-e06b6f33c36a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:58.965671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:58.967393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:58.967885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:58.968880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:58.970201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:58.970738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:58.976847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:58.996561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 184/366] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:58.998316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.010104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.010822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.011066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.011301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.011985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:59.012296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.013921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.031258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 183/367] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.032332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.108168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.143490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.215691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8955d2b-2208-41d2-b940-ec3a380532da None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.266755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0f2b0d-6849-45ec-951f-e9e841868f99 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.285712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.287151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.287857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.289137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.290596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:09:59.291903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.298014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.316967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 184/368] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.319287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.346010 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.347453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.347992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.348343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.353044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.353044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.355281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 185/369] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:09:58 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.357984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.546641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdeebfa-778e-4474-84c0-fec59a42b124 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.562549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc9e5f2-b182-41d6-bc09-294f3b71dc41 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.564203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.597770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:09:59.631834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.631834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.631834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.631834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.634343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1eadc2-2817-4208-9e71-e53e846b2363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.635373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eadc2-2817-4208-9e71-e53e846b2363 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.635827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eadc2-2817-4208-9e71-e53e846b2363 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.636254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1eadc2-2817-4208-9e71-e53e846b2363 None admin] PATH_INFO: `/v3/users/22872878dd2f4902949b1757b8f14a61` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.638251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eadc2-2817-4208-9e71-e53e846b2363 None admin] RBAC: Authorizing `identity:get_user(user_id=22872878dd2f4902949b1757b8f14a61)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.640730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1eadc2-2817-4208-9e71-e53e846b2363 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.644261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 186/370] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/users/22872878dd2f4902949b1757b8f14a61 => generated 293 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:09:59.649510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a0688-152f-49f9-a7bc-fbefb4cda788 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.649510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a0688-152f-49f9-a7bc-fbefb4cda788 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.649510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342a0688-152f-49f9-a7bc-fbefb4cda788 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.651402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 187/371] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:09:59 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:09:59.655949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a0688-152f-49f9-a7bc-fbefb4cda788 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.784105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:09:59.788527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:09:59.846749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342a0688-152f-49f9-a7bc-fbefb4cda788 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:09:59.919901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865a14da-d262-44ab-9e5d-e653c252b2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:09:59.920909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a14da-d262-44ab-9e5d-e653c252b2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:09:59.921722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a14da-d262-44ab-9e5d-e653c252b2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:09:59.922399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865a14da-d262-44ab-9e5d-e653c252b2c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:09:59.988012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 185/372] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 669 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:09:59.989802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:09:59.994908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.041824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.077359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a14da-d262-44ab-9e5d-e653c252b2c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.081739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865a14da-d262-44ab-9e5d-e653c252b2c8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.160681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0032e449-60b0-4332-92a7-efca54f602ef None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.221249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.224261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.224703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.225161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.226536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:00.227156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.229903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.242010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 186/373] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 260 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.243489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.266611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 188/374] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:00.270358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.270852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.271339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.309653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:00.312220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:00.437257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.438916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.448784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 189/375] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:00.450350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.479637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cf9a39-0c47-4ef6-8cf2-036fbf8ac98b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.578253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.579359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.579816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.580257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.583471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] RBAC: Adding query filter params (role.id=233bfd8b4640483288d9822efc76cb09, user.id=22872878dd2f4902949b1757b8f14a61, scope.system=all) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:00.584013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.586852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.623841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 187/376] 10.222.0.73 () {56 vars in 1308 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/role_assignments?user.id=22872878dd2f4902949b1757b8f14a61&scope.system=all&role.id=233bfd8b4640483288d9822efc76cb09 => generated 510 bytes in 381 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.625153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.638819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b589f3-8259-4b33-b725-dafa6d47beca None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.732348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.733045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.733323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.733594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.736104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.737773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.739615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 190/377] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.750201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:00.889038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a57ea95-fe06-4213-8119-9afccd4d4623 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:00.908762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:00.951159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:00.951817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:00.952091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:00.952358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] PATH_INFO: `/v3/projects/6d681c62c63c41f3b7033ae72341d972` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:00.953789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] RBAC: Authorizing `identity:get_project(project_id=6d681c62c63c41f3b7033ae72341d972)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:00.955655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:00.958297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 188/378] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/projects/6d681c62c63c41f3b7033ae72341d972 => generated 344 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:00.967835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.008191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5df598a-8094-4cd1-afc2-d88a925da586 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.013271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.119917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.122345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.122517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.123002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.125235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.128077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.137170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 191/379] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.138825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.161957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b142e1-b3a8-4c16-81c0-338188c3016f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.249621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.249621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.249621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.249621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] PATH_INFO: `/v3/groups/782b54f69f9241899916558448c24a0f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.250017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] RBAC: Authorizing `identity:get_group(group_id=782b54f69f9241899916558448c24a0f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.250017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.250017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 189/380] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:00 2022] GET /identity/v3/groups/782b54f69f9241899916558448c24a0f => generated 229 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:01.250017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.348554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f751438c-652a-4131-92c1-b8828e998396 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.372305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.414767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13084ef4-345a-424b-a66e-771037ded638 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.422858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.423992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.425710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.426232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.432590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.450237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:01.456619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:01.457651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:01.458433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:01.462722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 192/381] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:01.463271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.472778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.473793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.474295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.474817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.475801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.476303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.478762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.487915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 190/382] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 260 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.489614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.630333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d799cddf-99a9-4505-ad2b-23fd129db45f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.676714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.677387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.677661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.677928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.678617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.678943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.680365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d9026-add5-4ffa-9d00-1f96c2f74a08 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.692537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 191/383] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.694644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f13e-d5e3-4e4f-bfc4-9906c69b3eca None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.694935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f13e-d5e3-4e4f-bfc4-9906c69b3eca None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.695208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b0f13e-d5e3-4e4f-bfc4-9906c69b3eca None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.701127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 192/384] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:01.701127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0f13e-d5e3-4e4f-bfc4-9906c69b3eca None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:01.705947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:01.744931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f859745-5602-4928-b931-b95a18d07c62 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.793696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.794333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.794616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.794891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.795646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] RBAC: Adding query filter params (group.id=782b54f69f9241899916558448c24a0f, role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=6d681c62c63c41f3b7033ae72341d972) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:01.795950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.797442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492b2431-a746-45fc-9747-d3c40e8a34c7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.810846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 193/385] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/role_assignments?group.id=782b54f69f9241899916558448c24a0f&scope.project.id=6d681c62c63c41f3b7033ae72341d972&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 260 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.813150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.813439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.813739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.855749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:01.862662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:01.883343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b0f13e-d5e3-4e4f-bfc4-9906c69b3eca None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:01.964495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:01.967898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:01.969566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:01.969864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:01.971292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:01.973587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:01.975633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 193/386] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:01.984557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.046357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.078905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 194/387] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:02.141496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.199505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6142d314-879b-44ab-84ae-89d94c2780f4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.274930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.276102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.276613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.277160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.281241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.284665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.288457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 194/388] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:01 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:02.297680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.439059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.465178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d31d0-26a7-4480-b73c-f3f4e7e0219d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.566900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.569503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.576507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.577416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/groups/446fe6bcffc24f6193061610ba4e266b/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.586752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bd3cad3b8f49989950c0a9e6f1976c, group_id=446fe6bcffc24f6193061610ba4e266b, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:02.589919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:02.687696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0633a61-b8a5-4a35-9705-b18dc7941fe3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:02.700071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 195/389] 10.222.0.73 () {58 vars in 1442 bytes} [Tue Aug 23 22:10:02 2022] PUT /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/groups/446fe6bcffc24f6193061610ba4e266b/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:02.749824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:02.750500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:02.750784 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:02.751097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:02.764099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:02.770066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:02.775182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:02.778716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:02.780198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 195/390] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:02.794103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:02.935766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.002801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341525e2-8cd3-43b1-8853-a38dd5132dfc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.064518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.065975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.066257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.066534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.067420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:03.067727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.069621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.087816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 196/391] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:03.093261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.216819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d842715a-1425-4c88-9318-a6d29edc3e00 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.256836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.257415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.257688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.257961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.259184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.260764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.262833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 196/392] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:03.268204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.423840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619111d-14c5-4ed6-9d92-d3271cd6ffda None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.444951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.486751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1cb41-1494-412b-8ff1-2a85850a3efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.488255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1cb41-1494-412b-8ff1-2a85850a3efb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.488642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1cb41-1494-412b-8ff1-2a85850a3efb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.488989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1cb41-1494-412b-8ff1-2a85850a3efb None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.490287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1cb41-1494-412b-8ff1-2a85850a3efb None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.492087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1cb41-1494-412b-8ff1-2a85850a3efb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.494683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 197/393] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:03.500845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:03.516632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79ebb-979a-47b5-8e6d-a81ddf9195a0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.516998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79ebb-979a-47b5-8e6d-a81ddf9195a0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.517255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79ebb-979a-47b5-8e6d-a81ddf9195a0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.519906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 198/394] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:03.525385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.525777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.526013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.546429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:03.547634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:03.632232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 199/395] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:03.681969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:03.735518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f2164-ce61-487e-97fe-25019cd84f6f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:03.777939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d04dc15-9bcb-4cac-9500-8f8b93527b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:03.778527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04dc15-9bcb-4cac-9500-8f8b93527b72 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:03.778800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04dc15-9bcb-4cac-9500-8f8b93527b72 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:03.779078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d04dc15-9bcb-4cac-9500-8f8b93527b72 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/groups/446fe6bcffc24f6193061610ba4e266b/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:03.781410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04dc15-9bcb-4cac-9500-8f8b93527b72 None admin] RBAC: Authorizing `identity:create_grant(project_id=f6bd3cad3b8f49989950c0a9e6f1976c, group_id=446fe6bcffc24f6193061610ba4e266b, role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:03.782883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d04dc15-9bcb-4cac-9500-8f8b93527b72 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:03.862757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 197/396] 10.222.0.73 () {58 vars in 1442 bytes} [Tue Aug 23 22:10:03 2022] PUT /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c/groups/446fe6bcffc24f6193061610ba4e266b/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 0 bytes in 363 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:10:03.984819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.075347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3335ddb-b1de-43ae-b2e0-e4d91850679b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.075690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3335ddb-b1de-43ae-b2e0-e4d91850679b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.076071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3335ddb-b1de-43ae-b2e0-e4d91850679b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.077962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 198/397] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:04.083842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.084251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.084654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.104669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:04.106096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:04.108780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f4459-47ff-44c2-aa40-ebca999639e1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.126182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.169493 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.170399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.170903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.171396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.205089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:04.217081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:04.218109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:04.218941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:04.219808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 200/398] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:04.227352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.281165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 199/399] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:04.375284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77c39b5-86ce-4459-9cc3-4f3a9c42e3c4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.423982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.424974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.424974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.425378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.425886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:04.426222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.427618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.441777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 201/400] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:04.452158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.585567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.645227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f065b0f-a931-493a-836a-68c582a307ff None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.657044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.701351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.702017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.702301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.702701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.705513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.708875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.713136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 200/401] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:04.719734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.730434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.855952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6924097-fb0a-40c1-8476-3b9d167bc2eb None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.891493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11742b-86b5-4d21-b11b-f1a11419f5f2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:04.893594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bb332-5f5b-4205-bd23-025367a92acc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.894301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb332-5f5b-4205-bd23-025367a92acc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.894688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb332-5f5b-4205-bd23-025367a92acc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.894967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bb332-5f5b-4205-bd23-025367a92acc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.906266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bb332-5f5b-4205-bd23-025367a92acc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-192bb332-5f5b-4205-bd23-025367a92acc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:04.913074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:04.914100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:04.915334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 202/402] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:04.915988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45491984-5f94-401a-b98e-ca5f6e6c3067 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.916088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45491984-5f94-401a-b98e-ca5f6e6c3067 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.916683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45491984-5f94-401a-b98e-ca5f6e6c3067 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.921682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 203/403] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:04.922283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45491984-5f94-401a-b98e-ca5f6e6c3067 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:04.934908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:04.948509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7891920-4658-4b5c-bae5-cf4b85ef9a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:04.949571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7891920-4658-4b5c-bae5-cf4b85ef9a3f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.950051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7891920-4658-4b5c-bae5-cf4b85ef9a3f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.950614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7891920-4658-4b5c-bae5-cf4b85ef9a3f None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.952012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7891920-4658-4b5c-bae5-cf4b85ef9a3f None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:04.953788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7891920-4658-4b5c-bae5-cf4b85ef9a3f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:04.961257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 201/404] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:04.963006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:04.963493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:04.964298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:04.984423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:04.986027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:05.029636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.040764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45491984-5f94-401a-b98e-ca5f6e6c3067 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.073981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 202/405] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:05.081557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.082829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.082987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.083380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.083924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.084759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:05.085302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.086993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.098901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 204/406] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:05.104142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.261197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0fabe0-be14-4c9c-8ab4-903f0423e49a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.294880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.308948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.309614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df04112-7e28-4559-816d-28903eac87a8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.310969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.310969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.310969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/groups/446fe6bcffc24f6193061610ba4e266b/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.312833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] RBAC: Authorizing `identity:create_grant(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36, group_id=446fe6bcffc24f6193061610ba4e266b, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.314344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.367009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.367662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.367994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.368278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.369782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.371385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.373541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 205/407] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:05.380482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.408646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.418356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 203/408] 10.222.0.73 () {58 vars in 1442 bytes} [Tue Aug 23 22:10:05 2022] PUT /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/groups/446fe6bcffc24f6193061610ba4e266b/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:05.419678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.518987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b467dee4-64df-46ed-a69d-e551b28b90c0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.568233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.568899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.569270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.569546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.570935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.572453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.578930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63082d4b-4ec0-4d81-8c63-49c3ff72c754 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.580645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 206/409] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:05.586878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.636660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.637354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.637678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.638108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.657647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:05.668610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:05.669686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:05.670499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:05.671101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 204/410] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:05.679304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.716538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.731742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8b49e-cbbf-4fb8-9936-2a0393467c5d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.744282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:05.817036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371e8207-6022-4c4c-995a-97e4d896f380 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:05.821090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fde835e-ef91-4515-b4e0-b718efd8d486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.821883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde835e-ef91-4515-b4e0-b718efd8d486 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.822238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde835e-ef91-4515-b4e0-b718efd8d486 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.822589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fde835e-ef91-4515-b4e0-b718efd8d486 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/groups/446fe6bcffc24f6193061610ba4e266b/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.827930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde835e-ef91-4515-b4e0-b718efd8d486 None admin] RBAC: Authorizing `identity:create_grant(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36, group_id=446fe6bcffc24f6193061610ba4e266b, role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.829581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fde835e-ef91-4515-b4e0-b718efd8d486 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.861942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:05.863029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:05.863462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:05.863883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:05.864817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:05.865304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:05.866863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:05.877138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 205/411] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:05.889392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:05.926435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 207/412] 10.222.0.73 () {58 vars in 1442 bytes} [Tue Aug 23 22:10:05 2022] PUT /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36/groups/446fe6bcffc24f6193061610ba4e266b/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:10:06.011602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b411164-a05b-4472-8474-517ecb54f1e9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.011876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b411164-a05b-4472-8474-517ecb54f1e9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.012290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b411164-a05b-4472-8474-517ecb54f1e9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.018761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 208/413] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:06.027352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.027757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.027941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.070377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:06.071540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:06.086810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8dea65-83d7-4fac-ae3f-3a2c02138752 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.092802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.162700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.163431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.163712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.163989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] PATH_INFO: `/v3/groups/782b54f69f9241899916558448c24a0f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.165374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] RBAC: Authorizing `identity:get_group(group_id=782b54f69f9241899916558448c24a0f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.166802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.171271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 206/414] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:05 2022] GET /identity/v3/groups/782b54f69f9241899916558448c24a0f => generated 229 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:06.179442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:06.218593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 209/415] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:06.305367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4421a-d68e-44e7-948e-dcfcf7eedd28 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.343600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.344604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.345103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.345576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] PATH_INFO: `/v3/projects/6d681c62c63c41f3b7033ae72341d972` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.347791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] RBAC: Authorizing `identity:get_project(project_id=6d681c62c63c41f3b7033ae72341d972)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.349563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.351734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 207/416] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/projects/6d681c62c63c41f3b7033ae72341d972 => generated 344 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:06.357887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:06.517658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.566437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af509c0-ed05-4d9b-9176-38c10e344eff None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.654275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.655332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.655809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.656290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] PATH_INFO: `/v3/projects/6d681c62c63c41f3b7033ae72341d972/groups/782b54f69f9241899916558448c24a0f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.660440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] RBAC: Authorizing `identity:create_grant(project_id=6d681c62c63c41f3b7033ae72341d972, group_id=782b54f69f9241899916558448c24a0f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:06.663506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:06.669276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:06.746030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:06.774019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 210/417] 10.222.0.73 () {58 vars in 1442 bytes} [Tue Aug 23 22:10:06 2022] PUT /identity/v3/projects/6d681c62c63c41f3b7033ae72341d972/groups/782b54f69f9241899916558448c24a0f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:06.827447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560bf528-032c-4c0f-b89b-58ed590b3839 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:06.882650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0325dd37-4c63-4d7e-bdf7-e56c09fb5b57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:06.883360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325dd37-4c63-4d7e-bdf7-e56c09fb5b57 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:06.883757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325dd37-4c63-4d7e-bdf7-e56c09fb5b57 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:06.884035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0325dd37-4c63-4d7e-bdf7-e56c09fb5b57 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:06.897326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0325dd37-4c63-4d7e-bdf7-e56c09fb5b57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0325dd37-4c63-4d7e-bdf7-e56c09fb5b57 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:06.902717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:06.903677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:06.904497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 208/418] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:06.911449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:07.063141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c4beb-0b80-423e-a7dc-e56a8118f34e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:07.101541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:07.102528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1dc3b-c211-49cb-8210-6a6cc3bbe6d9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:07.103011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1dc3b-c211-49cb-8210-6a6cc3bbe6d9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:07.103108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d1dc3b-c211-49cb-8210-6a6cc3bbe6d9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:07.103178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:07.103568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:07.104039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:07.104800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 209/419] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:07.109044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] RBAC: Adding query filter params (name=member) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:07.109044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:07.109044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:07.114224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:07.114679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:07.117219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:07.139109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 211/420] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:10:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:07.148771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:07.150030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:07.161499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:07.174002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:07.300069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 210/421] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:07.411004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6dd29a-0c55-4e6e-81b5-99649d607068 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:07.478207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:07.479265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:07.479762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:07.480250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:07.641084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:07.643807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:07.647244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:07.736573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:07.829511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:07.895679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 212/422] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 736 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 23 22:10:07.902069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.013329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9c0de-fbd8-4ec6-8a48-31923cc468fd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.101218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56aeb64-5817-40ea-8a00-ced4690974e5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.103901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.128321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b210f60-e874-4ec7-bdae-a6d4931f13b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.129694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b210f60-e874-4ec7-bdae-a6d4931f13b5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.130159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b210f60-e874-4ec7-bdae-a6d4931f13b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.130632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b210f60-e874-4ec7-bdae-a6d4931f13b5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.151279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b210f60-e874-4ec7-bdae-a6d4931f13b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:08.158879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.159720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.160416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.160875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6b210f60-e874-4ec7-bdae-a6d4931f13b5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:08.161695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:08.162546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:08.165421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:08.166962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:08.167104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 211/423] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 428 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:08.167238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:08.167383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f36ae-ec48-448c-a59c-688dce24afbd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.167383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f36ae-ec48-448c-a59c-688dce24afbd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.167383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3f36ae-ec48-448c-a59c-688dce24afbd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.173050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 213/424] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:07 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:08.173327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 212/425] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:08.173478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.178901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f36ae-ec48-448c-a59c-688dce24afbd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.343597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.356528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3f36ae-ec48-448c-a59c-688dce24afbd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.358252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33613aed-3f85-4025-a7d1-622e42a55162 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.400849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1af6fa-cf58-4278-be0c-85121457c799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.401533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af6fa-cf58-4278-be0c-85121457c799 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.401816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af6fa-cf58-4278-be0c-85121457c799 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.402096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1af6fa-cf58-4278-be0c-85121457c799 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.403285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1af6fa-cf58-4278-be0c-85121457c799 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:08.405382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1af6fa-cf58-4278-be0c-85121457c799 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:08.407741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 213/426] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:08.409874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.410439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.410808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.431694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.432733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.432841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:08.433320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.433811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.434043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:08.435121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:08.435674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:08.438790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:08.456424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 214/427] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:08.458148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.525923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 214/428] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:08.533266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.538671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.627705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.640983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fd9fb4-8578-45cb-8c5f-9d0be2759cd7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.641904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2c28f9-7365-4bf0-96d8-822a0b26b94f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:08.680897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.681485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.681631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:08.681774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.682225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.682381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.682492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.682828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.683521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:08.683982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:08.685465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f7f7eb7-883f-4778-bc2b-761196319e21 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:08.693898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 215/429] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 611 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:08.695330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936759b-8108-4d34-a598-1200e4bddd5d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.695623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936759b-8108-4d34-a598-1200e4bddd5d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.695892 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8936759b-8108-4d34-a598-1200e4bddd5d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.697634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 216/430] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:08.703531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:08.703821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:08.704120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:08.725239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:08.726662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:08.765413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:08.766906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:08.810777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 217/431] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:08.873480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:08.887640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 215/432] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:08.897072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:08.941546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.036915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914b7be-b6b9-46c9-a2ca-8333488658f0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.074566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.075266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.075537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.075852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] PATH_INFO: `/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.077028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] RBAC: Authorizing `identity:get_project(project_id=f6bd3cad3b8f49989950c0a9e6f1976c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.078482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.080433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 218/433] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:08 2022] GET /identity/v3/projects/f6bd3cad3b8f49989950c0a9e6f1976c => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:09.089109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.132513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.234242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ecb48f-f5e4-40a4-ae61-06299f3e803f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.282844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.283502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.284413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.284680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.286283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.287917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.294687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 216/434] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:09.296007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.320312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.332389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a36de5-958d-4a70-8778-d8bcf3e02851 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.373554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.397215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.398347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.398827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.399305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.414368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:09.420226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:09.421212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 23 22:10:09.422009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:09.422617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 219/435] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:09.423415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.448280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22362e70-3bfc-4754-aef9-c1a54b164dcb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.504691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.505300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.505583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.505856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.506628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:09.507817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.509383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.520381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 217/436] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=f6bd3cad3b8f49989950c0a9e6f1976c&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 611 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:09.521819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.621762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6a1d32-1d1a-40ce-9bf0-4d0452ae023a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.676340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.677378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.677665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.678003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.693128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:09.699186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:09.700960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:09.702659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 220/437] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:09.707986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.729098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.762975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27025e21-ef21-4833-9994-94a7e34ceb19 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.792912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:09.814489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.815478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.815763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.816046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.816853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] RBAC: Adding query filter params (name=member) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:09.817202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.818608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea4b2d6-361a-4fa3-a178-c929a79d526e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.829515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 218/438] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:09.831887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fddce-22f7-49b8-a1a3-7b881fe2100f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.832390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fddce-22f7-49b8-a1a3-7b881fe2100f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.832793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9fddce-22f7-49b8-a1a3-7b881fe2100f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.834427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 219/439] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:09.838032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fddce-22f7-49b8-a1a3-7b881fe2100f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:09.860387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616801e-1947-49da-96f2-546d290c9232 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:09.911693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:09.913141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.913429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.913697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.914388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:09.914691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:09.916089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806b234a-52b4-40ca-bffc-d17681d2aeb8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:09.927374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 221/440] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:09.928673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:09.928951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:09.929245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:09.951082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:09.952343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:09.990826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9fddce-22f7-49b8-a1a3-7b881fe2100f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.033372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 222/441] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:10.033659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.034235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.037700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.038179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.038629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.046248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.102183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.125307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.127025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.192386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d8f23e-4a71-4e91-910d-4f9cef990c1a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.240087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.240704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.240991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.241295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.263751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 220/442] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:10.270065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.330116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.331676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.342024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.361482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.381319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321104e2-b214-448e-bc60-14531d87f36c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.421869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.422526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.422804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.423075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.424281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.425913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.427615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 221/443] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:10.428397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.473478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 223/444] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:10.474346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.538532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653e1a3c-b8cf-43e5-bb86-30edc68f18c6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.575422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.576073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.576353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.576627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.588585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:10.593609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:10.594719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 222/445] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:10.596101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.609373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.613248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55a359-8cd3-4708-9173-6dfdc8d6193d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.625618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.660908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.661739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.662075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.662483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.664009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.665659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.667921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 224/446] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:10.669117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.726448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd6ccc0-7784-4ece-8d51-b2dde13a860d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.767282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.767960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.768341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.768795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] PATH_INFO: `/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.770079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] RBAC: Authorizing `identity:get_project(project_id=2774fd9bf28d4ddfb8b3b6e48f84fa36)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.771545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.773263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 223/447] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/projects/2774fd9bf28d4ddfb8b3b6e48f84fa36 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:10.774032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.817504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf8b2de-e872-4808-97d3-558d766d993d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.863041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.863573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.863857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.864132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.864808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:10.865150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.866540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.875079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.875716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 225/448] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:10.876561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:10.903206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17765b2b-3a6e-4513-88b5-a3d1dfc66f4e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:10.931696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:10.945314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:10.946102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:10.946429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:10.946752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:10.947619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=87ab2740a8bb4a509daf398f95e522b1, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:10.947983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:10.949570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:10.958895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 224/449] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=87ab2740a8bb4a509daf398f95e522b1 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:10.959985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.023934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89db514c-3fe0-4026-ae8c-72f906f63cd2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.079970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.081176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.081757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.082227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] PATH_INFO: `/v3/groups/446fe6bcffc24f6193061610ba4e266b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.083956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] RBAC: Authorizing `identity:get_group(group_id=446fe6bcffc24f6193061610ba4e266b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.085915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.093426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 226/450] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/groups/446fe6bcffc24f6193061610ba4e266b => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:11.094825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.167716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1e235-d73b-4247-b202-ff526a914015 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.219280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.220959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.220959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.220959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.239434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.258520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4343c6b3-c3df-4564-8a36-ddb4dc681c59 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.262952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.311910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.312995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.312995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.313201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.313940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] RBAC: Adding query filter params (group.id=446fe6bcffc24f6193061610ba4e266b, role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:11.314492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.317415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.320625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.320625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.326667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 227/451] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/role_assignments?group.id=446fe6bcffc24f6193061610ba4e266b&scope.project.id=2774fd9bf28d4ddfb8b3b6e48f84fa36&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 611 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:11.483713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 225/452] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 520 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:11.485337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.651812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dbabe0-249c-4525-9b9e-b48c84e16bd5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.703115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.704178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.704655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.705165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] PATH_INFO: `/v3/projects/6d681c62c63c41f3b7033ae72341d972` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.707916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] RBAC: Authorizing `identity:get_project(project_id=6d681c62c63c41f3b7033ae72341d972)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.710678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.713690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 228/453] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/projects/6d681c62c63c41f3b7033ae72341d972 => generated 344 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:11.719945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.772965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.837410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6db8a2-b5eb-4154-858e-c9cde2ce4bc7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:11.875543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b9c540-915d-4c5b-9753-a444ef983b62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:11.878594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9c540-915d-4c5b-9753-a444ef983b62 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:11.878594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9c540-915d-4c5b-9753-a444ef983b62 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:11.878594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b9c540-915d-4c5b-9753-a444ef983b62 None admin] PATH_INFO: `/v3/groups/782b54f69f9241899916558448c24a0f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:11.878594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9c540-915d-4c5b-9753-a444ef983b62 None admin] RBAC: Authorizing `identity:get_group(group_id=782b54f69f9241899916558448c24a0f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:11.879636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b9c540-915d-4c5b-9753-a444ef983b62 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:11.881477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 226/454] 10.222.0.73 () {56 vars in 1180 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/groups/782b54f69f9241899916558448c24a0f => generated 229 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:11.888409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:11.928575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:11.992982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2070bd6c-81cd-44b6-9245-2f71ce2a9090 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:12.028400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:12.028925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:12.029231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:12.029501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:12.030238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] RBAC: Adding query filter params (group.id=782b54f69f9241899916558448c24a0f, role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=6d681c62c63c41f3b7033ae72341d972) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:12.030538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:12.031965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8aac05-1fb7-410c-9189-66774d4f08ca None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:12.039069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 229/455] 10.222.0.73 () {56 vars in 1376 bytes} [Tue Aug 23 22:10:11 2022] GET /identity/v3/role_assignments?group.id=782b54f69f9241899916558448c24a0f&scope.project.id=6d681c62c63c41f3b7033ae72341d972&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 611 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:14.489031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438975f4-cf83-40ee-b39d-ed31a2127008 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:14.489566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438975f4-cf83-40ee-b39d-ed31a2127008 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:14.489566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438975f4-cf83-40ee-b39d-ed31a2127008 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:14.491192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 227/456] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:14.496485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209697f2-3393-4252-8e21-f72d51f5ff48 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:14.497329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209697f2-3393-4252-8e21-f72d51f5ff48 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:14.497772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209697f2-3393-4252-8e21-f72d51f5ff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:14.530544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-209697f2-3393-4252-8e21-f72d51f5ff48 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:14.532030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-209697f2-3393-4252-8e21-f72d51f5ff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:14.639806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 230/457] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:15.026356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438975f4-cf83-40ee-b39d-ed31a2127008 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.062258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37e3d-8f65-4585-a3fc-ac0487f2587a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.062935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37e3d-8f65-4585-a3fc-ac0487f2587a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.063432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf37e3d-8f65-4585-a3fc-ac0487f2587a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.066957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 231/458] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:15.073197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.073785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.074268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.110147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:15.111966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:15.169414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.188352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438975f4-cf83-40ee-b39d-ed31a2127008 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.206360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.229434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314add52-f475-4165-9222-f7221bf0131c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.230448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314add52-f475-4165-9222-f7221bf0131c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.230925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314add52-f475-4165-9222-f7221bf0131c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.231508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314add52-f475-4165-9222-f7221bf0131c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.249106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314add52-f475-4165-9222-f7221bf0131c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.250916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314add52-f475-4165-9222-f7221bf0131c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.256746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 232/459] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-314add52-f475-4165-9222-f7221bf0131c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:15.270561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:15.271546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:15.272442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:15.272442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:15.272442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:15.274660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 228/460] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:15.282560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.332952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85349c-1401-4f44-8d03-a581d1a9956a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.333265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85349c-1401-4f44-8d03-a581d1a9956a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.333706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85349c-1401-4f44-8d03-a581d1a9956a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.335488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 229/461] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:15.341221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c87eb-0ece-4464-b80d-8d070ff3714f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.341752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c87eb-0ece-4464-b80d-8d070ff3714f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.342178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015c87eb-0ece-4464-b80d-8d070ff3714f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.366116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-015c87eb-0ece-4464-b80d-8d070ff3714f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:15.367665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-015c87eb-0ece-4464-b80d-8d070ff3714f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:15.443125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db51dcbd-a2d2-43f3-8488-e13a0a62f9bb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.491175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.492761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.493571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.498090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.498090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:15.498090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.499525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 230/462] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:15.499941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.501651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed03e1-4394-4e93-81a8-55301853ebfc None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.502761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed03e1-4394-4e93-81a8-55301853ebfc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.503575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed03e1-4394-4e93-81a8-55301853ebfc None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.505821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 231/463] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:15.515402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.516088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.516737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.550808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 233/464] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:15.560065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.576093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:15.585944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:15.615430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.675078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 232/465] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:15.679641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:15.719581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:15.796932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc48fb7-f951-4f5c-8654-702a04498be2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.891506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.893578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.894974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.895683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.896728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.899947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:15.920217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13241ec1-ffe5-4b9d-afef-7a5885f68937 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:15.972330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:15.973179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:15.973457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:15.973727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:15.985569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:15.988095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:16.013758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:16.021791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:16.024110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:16.024110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:16.024110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:16.024110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 233/466] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:16.024110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.028776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 234/467] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:15 2022] POST /identity/v3/users => generated 246 bytes in 469 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:16.029284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.166066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.200879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.227771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4e2fe3-e38e-444a-8889-64bae55a5973 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.242613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275b3be4-83c7-425a-9640-2dbb5f2c4289 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.296810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.297957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.298519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.298878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.308444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.309971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.310447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.311355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.318064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.319536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.332048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.334829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:16.335132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:16.335989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:16.336820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:16.336820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:16.336820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:16.336820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 234/468] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:16.337573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-17882321-ee48-4352-bb72-955092e4cf9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:16.360782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:16.362065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:16.363104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:16.363104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:16.363104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:16.363104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 235/469] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:16.363789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de551-711d-4b6b-96e7-4f50baa8cfe2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.364082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de551-711d-4b6b-96e7-4f50baa8cfe2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.364353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12de551-711d-4b6b-96e7-4f50baa8cfe2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.373097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 236/470] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:16.373097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de551-711d-4b6b-96e7-4f50baa8cfe2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.504739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac9a752-feaf-49d5-a27c-a90ee5a6ee66 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.547195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.550428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12de551-711d-4b6b-96e7-4f50baa8cfe2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.564731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.566720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.567360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.567718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.568721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:16.569246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.571788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.605336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.613371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 235/471] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:16.613371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.623718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.630836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.630836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.630836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.630836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:16.630836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.630836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31447079-1d99-4216-9b1f-85e013687839 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.653064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 237/472] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:16.653064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.653064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.653064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.672342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:16.673670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:16.761125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 238/473] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:16.761125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.790930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b13b40-8b9b-4551-a5f9-ce78fef2d597 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.840248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.840959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.841322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.841656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.842447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:16.842822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.844390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:16.857286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 236/474] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:16.862805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:16.918861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad860b2b-1932-4b8d-aefd-4ec2d1087599 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.919902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.954891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:16.993375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:16.994597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:16.995044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e49405-db73-4bad-a60f-afa1feba7d1a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:16.995178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:16.995525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:16.996603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:16.999613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.039721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.040683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.041180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.041614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.042417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.044038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.109046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 239/475] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:16 2022] POST /identity/v3/users => generated 248 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:17.109046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.134046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 237/476] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:16 2022] POST /identity/v3/users => generated 248 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:17.178843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.355280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0600bd-6557-477e-999a-020f46d151fd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.384786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc6662d-79be-49b5-b24e-757742e254ba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.388888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:17.400749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.402839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.402839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.402839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.402839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.404626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.438739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.439358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.440182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.440451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.441249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:17.442645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:17.479022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.sql.core [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 22:10:17.479022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 22:10:17.479022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [parameters: {'id': '1eeb7675e45848408b69af2af07de6f3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 22:10:17.479022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81612) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 22:10:17.497544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 240/477] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:17 2022] POST /identity/v3/users => generated 249 bytes in 395 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 23 22:10:17.500512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return ref.to_dict() Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:10:17.502872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application yield resource Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise self.value Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application session.commit() Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 22:10:17.504626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.flush() Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application compat.raise_( Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 22:10:17.507198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:10:17.508943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:10:17.510767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [parameters: {'id': '1eeb7675e45848408b69af2af07de6f3', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:17.512535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 23 22:10:17.525474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:17.526980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 238/478] 10.222.0.73 () {60 vars in 1131 bytes} [Tue Aug 23 22:10:17 2022] POST /identity/v3/roles => generated 144 bytes in 323 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:17.527098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.592211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:17.742704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d88127-47b9-421a-adbb-9f1c097a5d5b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:17.785753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa3e3f5-c6bf-4be9-b835-47e5f77ff166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:17.786553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3e3f5-c6bf-4be9-b835-47e5f77ff166 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:17.786930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3e3f5-c6bf-4be9-b835-47e5f77ff166 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:17.787319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3e3f5-c6bf-4be9-b835-47e5f77ff166 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:17.800065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa3e3f5-c6bf-4be9-b835-47e5f77ff166 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4fa3e3f5-c6bf-4be9-b835-47e5f77ff166 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:17.805784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:17.806882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 23 22:10:17.808058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:17.809959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 241/479] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:17.820218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:17.966344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61bb970-77ed-4e13-aeca-8f0504e670bf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:18.015662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:18.016381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.016757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.017157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.018629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:18.019280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:18.020861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ac9fa3-02e3-45c7-b369-a6d34d229377 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:18.038075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 239/480] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:18.308635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2178-3a87-42d8-aac7-f13b6b6b89af None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.309047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2178-3a87-42d8-aac7-f13b6b6b89af None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.309441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dd2178-3a87-42d8-aac7-f13b6b6b89af None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.311214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 242/481] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:18.326053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:18.326053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:18.326053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:18.354328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:18.356547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:18.398446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:18.477363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 240/482] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:18.918580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd2178-3a87-42d8-aac7-f13b6b6b89af None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.061352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.081484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dd2178-3a87-42d8-aac7-f13b6b6b89af None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.147476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.148396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.148853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.149379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.169125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:19.178008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:19.178940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:19.179723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:19.180799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 243/483] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:19.186250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.366946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2058193a-6972-49c7-9a59-f5671ed887a7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.431993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.432547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.432827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.433130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.433830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:19.434143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.435590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158579ad-b875-49be-b37d-f5ca31fa90e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.449519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 241/484] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:19.456072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.631201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5a71-2d78-4b1e-97bb-84c77961f2c5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.631554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5a71-2d78-4b1e-97bb-84c77961f2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.631725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5a71-2d78-4b1e-97bb-84c77961f2c5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.633104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 242/485] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:19.639788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d73f1-a96b-46c4-a56a-65e920d368d0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.640261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d73f1-a96b-46c4-a56a-65e920d368d0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.640759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d73f1-a96b-46c4-a56a-65e920d368d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.649076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37128a49-fc49-439e-85ab-a56d8cfc40a1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:19.693787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-393d73f1-a96b-46c4-a56a-65e920d368d0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:19.695698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-393d73f1-a96b-46c4-a56a-65e920d368d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:19.698779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:19.699571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.699844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.700115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.717741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:19.719230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:19.727175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:19.749991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:19.752554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:19.754618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:19.754618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:19.754618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:19.754618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 244/486] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:19.758429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:19.762999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:19.900282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 243/487] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:19.985351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745321b-e6cf-4517-a84e-0b1b54764fb1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:19.985982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745321b-e6cf-4517-a84e-0b1b54764fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:19.987098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745321b-e6cf-4517-a84e-0b1b54764fb1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:19.989743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 244/488] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:20.003454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.003772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.004011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.043278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd44c771-91c3-485e-88b5-76dff00a4ace None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.055553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:20.057621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:20.136951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.138005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.138454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.138904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.139723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.140205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.141791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.169993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 245/489] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:20.185779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.187109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 245/490] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:20.281464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.365380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.452503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ab4502-9839-41c0-bf7c-e3c231992e49 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.465421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.514810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.515296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7338c82f-719b-4924-92af-e3b8483b69f4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.515534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.515817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.516085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.530280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.533852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.560234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.561258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.562269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.562741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:20.566198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:20.567092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:20.567980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:20.573208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 246/491] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:20.574688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.578147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:20.584786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:20.585901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:20.586670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:20.589185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 246/492] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:20.590195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.722838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdcab4bd-9c72-409b-9f14-cbf3584d1902 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.725976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb1c23-e892-46d5-9f8f-1f1f9d1711bf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.755775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.769259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.771351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.771351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.771351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.771972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.773003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.773532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.774065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.775440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.775995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.778625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e33af9b-fa07-4702-9b29-8945fa40b2f1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.782124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:20.788770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:20.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:20.790533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:20.794458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 247/493] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:20.795517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.825096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:20.844719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 247/494] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.844719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b993da5-f15c-45ce-ad29-d925f1b8acba None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.844719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b993da5-f15c-45ce-ad29-d925f1b8acba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.844719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b993da5-f15c-45ce-ad29-d925f1b8acba None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.844719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 248/495] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:20.844719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b993da5-f15c-45ce-ad29-d925f1b8acba None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.921895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3a100-dc05-4839-b838-865b02f73f29 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:20.961065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:20.961698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:20.961979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:20.962250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:20.962930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:20.963236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:20.964634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:20.975827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 248/496] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:20.977215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:20.995555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b993da5-f15c-45ce-ad29-d925f1b8acba None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.056088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.067396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.068026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.068782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.068782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.069378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:21.069748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.071220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1a67a5-a871-4f13-aab6-76eadd97236f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.081844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 249/497] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:21.088469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.088469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.088469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.109547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:21.110835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:21.125535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5a766-85f1-4896-9683-9469428061e4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.167535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.168819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.169341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.169881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.176952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.188846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 250/498] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:21.202007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.253978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.255832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.311579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.352779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa49c2d-4c2c-499e-b48d-5b02842c97ba None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.384583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 249/499] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:21.386031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.391851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.392553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.393471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.393976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.407307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.409280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:21.426516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:21.427464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:21.428302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:21.428302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.428302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:21.428302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 251/500] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:21.428515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.494146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.538006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8b248f-44ae-4ad5-8b44-8b0e9bf9eda3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.590125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b63dd-dd26-4076-9043-8f71995a4cd0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.606227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.606938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.607179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.607452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.625222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.627787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.629555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.630588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.631043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.631484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.632019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.647010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.649050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:21.649551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:21.650467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:21.651319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:21.651319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.651319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:21.651319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 250/501] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:21.651537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-99ee26d3-7493-467e-84bd-83e1fe63cb04 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:21.670406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:21.671540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:21.672543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:21.672543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:21.672543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:21.672543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 252/502] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:21.672812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d84701-096e-4c06-b1bd-a19c951458f7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.673320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d84701-096e-4c06-b1bd-a19c951458f7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.673320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d84701-096e-4c06-b1bd-a19c951458f7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.679536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 253/503] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:21 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:21.680459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d84701-096e-4c06-b1bd-a19c951458f7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.817180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:21.819430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541f22ec-1380-4f09-b1ec-029e298aad69 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.824990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d84701-096e-4c06-b1bd-a19c951458f7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:21.859071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.859990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.860478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.860929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.862648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:21.863111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.864202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:21.864910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.865309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.865470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.865730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.866475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:21.866874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:21.868440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dec6b5-a05c-4033-9761-11bb8b0f6c60 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:21.884059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 254/504] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:21.889661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 251/505] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:21.889839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:21.889930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:21.890039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:21.890039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:21.914116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:21.915420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:21.961162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.010749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 255/506] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:22.011621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.019771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1456a1c0-0f5a-4970-a1e0-4da55c68be29 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.081650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.082670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.083160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.083791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.084970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:22.085600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.088110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.115675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.115865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 252/507] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:22.115865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.134237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a55f73d-bee8-4584-bd58-4115ff2af49f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.173165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.174352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.174641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.174914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.188578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:22.193953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.196697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 256/508] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.197566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.238937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c622a7-2f41-4405-8a7b-0f503673f2f7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.259714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.288028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.288676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.288947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.289254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.298153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.300500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:22.312772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.314040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:22.315061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:22.317106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 253/509] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.318766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.353594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a7a64f-d6d9-4706-9112-efc559f19c59 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.399617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.400171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.400446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.400717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.411481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.412954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:22.426551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.433595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 257/510] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.434668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.444044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8db3a67-f751-43ab-882f-5dbba457f8d3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.481027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.481696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.481974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.482247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.493395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.494873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:22.507871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.509678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 254/511] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.510965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.554609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.563494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b82ad89-edbd-484f-b998-8972ceec7190 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.607110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.607732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.608017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.608294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.608976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:22.609318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.610761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.620807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 258/512] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:22.621701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.634918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d54a5da-eac8-453e-8a95-a4fb353bffd3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.670760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.671297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.671576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.671847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.672876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:22.672876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.676541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.697548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.725375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 255/513] 10.222.0.73 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:22.726411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.750728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5dfdf3-dfdb-4711-82fb-99d5c5a0056e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.786892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.787541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.787818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.788094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.800137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.801666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:22.817360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:22.820809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:22.822710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:22.822710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:22.822710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:22.822710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 259/514] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:22.822710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.844360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:22.852285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c20097f-100d-4755-9c1b-ab39b7e362ec None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.895674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:22.896543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:22.896880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:22.897258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:22.898106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:22.898417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:22.899967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:22.921921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 256/515] 10.222.0.73 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:22.924454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:22.956172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648ddb64-40db-4c16-9d7c-7df3a4468549 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:22.977656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.001181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.001908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.002351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.002634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.003404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.003749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.005286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.034451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 260/516] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:23.034451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.049455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e357c7-6dba-4b12-9b32-9514127c4225 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.095225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.096623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.096623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.096623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.108147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.109814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:23.130066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:23.131255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:23.132107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:23.132107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:23.132107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:23.132107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 257/517] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:23.132107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.142554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.167681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d49336-ebd6-4568-82fe-a51069c2e639 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.204714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.205584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.205757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.205890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.266431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3d4b48-c2e9-435f-8cc4-b9f128f2362a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.276805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.304019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.305109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.311032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.311826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.312168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.312505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.313272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.313657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.316076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.328960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 258/518] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:23.329945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.416480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.441428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 261/519] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:23.442332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.463027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80be777-af39-424d-bbbe-343193d3cbf1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.507798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.508425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.508697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.508973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.558946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.585616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.587086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.592965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd3daa2-837a-441b-bc8a-3332f544cf33 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.632269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.632890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.633211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.633490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.634236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=059f72c1793e492fb052bebbf29b5bec) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.634541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.636022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.641834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 262/520] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/role_assignments?user.id=059f72c1793e492fb052bebbf29b5bec&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:23.642868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.672328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.731837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 259/521] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:23.732836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.787896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04847b6f-6063-4c25-90a4-b19c83b21174 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.838634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.839276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.839560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.839836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.840515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:23.840822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.842293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.857903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 263/522] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:23.858787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:23.862710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:23.881332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b36fa-499f-40eb-b5a5-660da93d16c4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:23.926317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:23.926948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:23.927228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:23.927503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:23.928553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:23.930913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:23.980676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0369263d-2903-496a-949a-b229c7f7dfa9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.034741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.035451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.035733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.036007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.053048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 260/523] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:23 2022] POST /identity/v3/users => generated 247 bytes in 314 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:24.053048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.061890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.064499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.091861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:24.092534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.093441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:24.094413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:24.094413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:24.094413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:24.094979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 264/524] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.096663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.205529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.214414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de90758c-697b-4ada-a519-73a8cb4f50ef None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.275606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.276269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.276551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.276820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.287582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd756fd-fa3a-4ae1-95f0-4469735d2579 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.295930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.297934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:24.319532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.320513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:24.321418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:24.321418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:24.321418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:24.325154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 261/525] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.326064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.334784 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.335434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.335718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.335993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.346380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.347862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:24.360089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.361196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:24.362609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 265/526] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.363169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.419792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.452726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d308a4f-cb33-4531-b7ac-8b0e46409b21 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.495104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.495726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.496008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.496285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.496947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:24.497338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.499353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.512403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846b035b-4e06-4107-9fc6-581b9460690b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.518023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 262/527] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:24.521925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.536440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.562030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.562858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.563286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.563659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.564521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:24.564981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.566525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.582056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 266/528] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:24.583168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.678226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40146abe-17b5-41f1-b117-d8a0ef0e797b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.716421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749b6340-e035-4ef1-980e-87d41740e64d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.723751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.730640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.731902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.732433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.732852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.733930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:24.744282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.744282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.758430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.759179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.759547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.759829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.765922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 263/529] 10.222.0.73 () {56 vars in 1139 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:24.767163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.768137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.769940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:24.782500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.784734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 267/530] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.786197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:24.825111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:24.899445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c44baa-7d78-4cc4-9723-4cf39ee23deb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.904492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f233eee3-2cf1-460a-9a49-a26de8bf4755 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:24.943306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.944326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.944742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.945183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.947234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:24.948034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:24.948457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:24.948881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:24.957320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:24.959024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:24.972226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:24.973174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:24.974022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:24.976519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 264/531] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:24.977742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.033617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.036757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.039347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.076722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.091888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed1f352-fc41-4b96-9db7-36e7b18bda9a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.135160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.136146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.136623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.137195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.138233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.138781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.140423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.166506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 268/533] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:25.166506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.166966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 265/533] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.168187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.279525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f1bd31-5407-4d9a-a5f9-643fb258ff9e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.285942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e62ca284-fa4a-44d0-ad9c-4fb4a0a84068 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.294509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.327775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.328653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.329085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.329479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.330293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.331282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.331720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.332145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.333033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.333713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.336313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.344648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.346432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.347675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.356452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 269/534] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.357728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:25.367428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:25.374611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:25.375605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:25.375605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:25.375605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:25.375605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 266/535] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:25.375605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.493424 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84621d95-0505-45d8-82ca-5c8b80662209 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.497905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b55d4b-16cb-468c-9735-18df59024cfe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.533847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b392c-3640-407f-8728-be2edf19518d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.535418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b392c-3640-407f-8728-be2edf19518d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.535914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b392c-3640-407f-8728-be2edf19518d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.536375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958b392c-3640-407f-8728-be2edf19518d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.537328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b392c-3640-407f-8728-be2edf19518d None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=c498027e07ce4bc28bc0518f22047b16) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.537822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b392c-3640-407f-8728-be2edf19518d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.538231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.538913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.539263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.539606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.540324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958b392c-3640-407f-8728-be2edf19518d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.540441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=f17bae22fe2b4aa09619bdc0b3e0b4a3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.540795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.542357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.547834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 270/536] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/role_assignments?user.id=c498027e07ce4bc28bc0518f22047b16&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.549025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b392c-3640-407f-8728-be2edf19518d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.561069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 267/537] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/role_assignments?user.id=f17bae22fe2b4aa09619bdc0b3e0b4a3&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.633924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.688216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958b392c-3640-407f-8728-be2edf19518d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:25.735919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:25.737852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.738415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.738869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.739771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:25.740259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:25.741929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe1f33-2639-4dcf-9f74-b621ddffc417 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:25.758787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 271/538] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:25.765622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:25.823141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:25.829868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f894d9-c236-4ca6-9faf-510f95131f92 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.830248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f894d9-c236-4ca6-9faf-510f95131f92 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.830403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f894d9-c236-4ca6-9faf-510f95131f92 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.832077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 272/539] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:25.837671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:25.837951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:25.840106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:25.861814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:25.863073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:25.962142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 273/540] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:26.003822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e4abb-e6e2-4eae-bd03-5d8a3fce56a7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.056594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.057278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.057561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.057837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.066710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.068270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:26.086052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:26.087629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:26.088536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 268/541] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:26.099888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.180700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.244795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9268ee53-eaea-4e75-ab01-fdb945f829ae None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.284123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.286279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.286279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.286279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.287612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:26.288130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.289932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.305768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 274/542] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:26.312956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.387057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.456215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.476821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be97a44-9ee6-4bb1-a4da-fa440cc16503 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.530707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.531217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.531496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.531766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.532817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=b05fdc39b370474fa59a63b4b0c7748a) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:26.533865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.536135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c220138-aa77-4d21-a3ff-8d48e6493259 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.546972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 269/543] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/role_assignments?user.id=b05fdc39b370474fa59a63b4b0c7748a&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:26.559760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d088a-94d8-4fd8-b3cc-683e2a80166d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.560827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d088a-94d8-4fd8-b3cc-683e2a80166d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.561141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4d088a-94d8-4fd8-b3cc-683e2a80166d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.565935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996d7ef5-cf0f-4ca0-9e92-a421ef123b48 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.573053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 270/544] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:26 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:26.579066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.579401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.579612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.590602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:26.625184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:26.626453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:26.639046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.639715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.639998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.640272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.667188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:26.672070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:26.674174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:26.675711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 275/545] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:26.682943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:26.818471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 271/546] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:26.914993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356abb7-e99c-4508-ba20-7f315fc5e09a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:26.975697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:26.977046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:26.977776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:26.978487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:26.979845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:26.980687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:26.983489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:26.999448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 276/547] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:27.005533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.164433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.182120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.196965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7db949-b094-4e7e-9813-c3b35ab1c261 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.266468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.286830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.287907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.288378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.288878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.329610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.332259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:27.364440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:27.366919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:27.369969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:27.369969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:27.369969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:27.371609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 272/548] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 366 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:27.382341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.444434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231e610-ea03-483e-8670-bc5de3ef4a7e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.520005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.520740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.521096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.521431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.549405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:27.554492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:27.555389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:27.556183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:27.557320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 277/549] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:27.566252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.610173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d52e98-c708-4bb9-a513-472bd78013cf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.655381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.661936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.673189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.673189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.673189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.673189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:27.673189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.673189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.681663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 273/550] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:27.686711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:27.688711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.710673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0695b5b-74a5-4700-88cb-c9584fbe2b47 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:27.772094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:27.773046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:27.773544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:27.774000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:27.774873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:27.775382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:27.777174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:27.788189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 278/551] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:27.798089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:27.923698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec397e2-8d26-4b02-a416-6842a84b6041 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.013987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.014558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.015066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.015340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.029847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.031321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.053938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da864ef4-bb67-47ac-a1bd-13bb17d689f4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-86d366fe-1f47-4fe0-833e-e8125e640631 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:28.060475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.061450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:28.062437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:28.063127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 274/552] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.067070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a46186-d772-46b4-ae2c-3d5ab371d32e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.067459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a46186-d772-46b4-ae2c-3d5ab371d32e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.067803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a46186-d772-46b4-ae2c-3d5ab371d32e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.069805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 275/553] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:28.071867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a46186-d772-46b4-ae2c-3d5ab371d32e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.106094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.106774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.107143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.107420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.121441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.122974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9c7d0718-1fb7-4cce-9ff6-24fef71eeac6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:28.139386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.140383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:28.141785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:28.141785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:28.141785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:28.141785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 279/554] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.142302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e2784-7fef-4962-aafb-37159bbe3b82 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.142590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e2784-7fef-4962-aafb-37159bbe3b82 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.142862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1e2784-7fef-4962-aafb-37159bbe3b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.146357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.151316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.163505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ba1e2784-7fef-4962-aafb-37159bbe3b82 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:28.164735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ba1e2784-7fef-4962-aafb-37159bbe3b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:28.202616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a46186-d772-46b4-ae2c-3d5ab371d32e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.243654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.244266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.244544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.244812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.245859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.246163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.253447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 280/555] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:28.253628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd71022f-d3e5-4509-aeaa-e306356c6db1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.253707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4951666-aa0c-4b62-9377-4d2c6bdac70d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.253707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4951666-aa0c-4b62-9377-4d2c6bdac70d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.253707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4951666-aa0c-4b62-9377-4d2c6bdac70d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.254110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 281/556] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:28.255444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4951666-aa0c-4b62-9377-4d2c6bdac70d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.268868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 276/557] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:28.270164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.270453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.270724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.293195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:28.295310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:28.401055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 277/558] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:28.401055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.423946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4951666-aa0c-4b62-9377-4d2c6bdac70d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.495361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.506013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.506648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.506911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.511692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.511692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.511692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.511692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.514482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.532419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 282/559] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:28.539865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.598387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da98fe-df39-42c9-a54c-b90c7c337928 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.636464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.637137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.637424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.637696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.649389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.650888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.657054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9714ca-b551-400d-98d1-25932f6abeb9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:28.668019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.669143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:28.670027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:28.670027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:28.670027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:28.675209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 278/560] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.676157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.696540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.697281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.697663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.697944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.707492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.709311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:28.729289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.730492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:28.731757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 283/561] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:28.732473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.791663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.803537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77585a31-f2a7-4948-a341-efd783c651b8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.813684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:28.858237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.858872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.859154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.859460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.868988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1dfe0ff-46c0-41b0-a87f-b659d3a15ec8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:28.873927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:28.879472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:28.880460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:28.881363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 279/562] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:28.882117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:28.909583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:28.911708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:28.911708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:28.911708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:28.911708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:28.911708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:28.913216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:28.927233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 284/563] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:28.928148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.002126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d8a4ed-8a36-4784-be3e-d011c78d49e3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.042637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efbbc36-d6ab-4a4d-925e-a26733b1d8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.043181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbbc36-d6ab-4a4d-925e-a26733b1d8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.043453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbbc36-d6ab-4a4d-925e-a26733b1d8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.043814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efbbc36-d6ab-4a4d-925e-a26733b1d8c4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.045912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1f93bd-a93c-41a5-88c9-0fce43c712ba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.054770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efbbc36-d6ab-4a4d-925e-a26733b1d8c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8efbbc36-d6ab-4a4d-925e-a26733b1d8c4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:29.059953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:29.061061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:29.061907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 280/564] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:29.062582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f155c8a-a2b9-4788-afe4-bff771b4a1ca None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.062582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f155c8a-a2b9-4788-afe4-bff771b4a1ca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.062760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f155c8a-a2b9-4788-afe4-bff771b4a1ca None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.063764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.065206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 281/565] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:29.065916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f155c8a-a2b9-4788-afe4-bff771b4a1ca None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.081421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.090727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.091624 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.091897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.092170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.093439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.093439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.095798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.115149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 285/566] 10.222.0.73 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.117285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.174368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f155c8a-a2b9-4788-afe4-bff771b4a1ca None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.209629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.210238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.210517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.210786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.211446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.211744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.213136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.219869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e407aac-0e6e-404c-968a-cf1d21529e77 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.224783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 282/567] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.224783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.257397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.258031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.258311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.258583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.266506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.268652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-69648ed4-170c-4feb-89a5-439d69c085ed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:29.280807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:29.281767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:29.282702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 286/568] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:29.283246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.283537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.283627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.302325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:29.303466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:29.308676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.334792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b1e169-ac2b-4bc4-b546-3549a77f200a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.336021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.369922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.370535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.370818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.371102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.371770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.372071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.373471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.380236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 287/569] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:29.381461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.382527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 283/570] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.383351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.502846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8612775-6457-47b7-bc8b-c8db2d88d425 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.507947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a7c0a0-be83-4697-b32d-73775d5d1130 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.540330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.540969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.541274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.541542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.544618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.550330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.554173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.554323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.554595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.554874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.556411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:29.571230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:29.572262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:29.573174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:29.573174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:29.573174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:29.577227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 284/571] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:29.578105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.593472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.639821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.641522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.685049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf904d-c448-4193-a6da-e9e2723ce4d8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.722784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.723489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.723772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.724043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.724742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.725081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.726509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.739484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 285/572] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.740355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.753071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 288/573] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:29.753928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.801222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.824655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:29.852670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ad43a4-e43a-4fb3-98d1-2e84fbbd5bb8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.865794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8fc8501-6544-48ff-b313-a3cfc96cfc38 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:29.887007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.887631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.887917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.888190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.898904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.900304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:29.900433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.900922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:29.901223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:29.901494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:29.902166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:29.902459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:29.903806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:29.916414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:29.917709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:29.918889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:29.918889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:29.918889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:29.918889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 286/574] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:29.918889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:29.925906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 289/575] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:29.926145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.019761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.022816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931f41e3-cd5d-416a-9692-e901123d2949 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.028325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fe466-67e5-4d68-b8d3-d797448330ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.052703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.058368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.059003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.059270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.059523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.064143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.064789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.067788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.067788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/059f72c1793e492fb052bebbf29b5bec/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.070711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:30.076103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.077227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 287/576] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.078514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.083371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=059f72c1793e492fb052bebbf29b5bec, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.085144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.184931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fd58a9-2760-4e3a-a9bb-f0b0349f2261 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.186629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 290/577] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:29 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/059f72c1793e492fb052bebbf29b5bec/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:30.187495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.227281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.228299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.228787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.229104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.242980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.245049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:30.263510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:30.265993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:30.265993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:30.265993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:30.265993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 288/578] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.265993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.288581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.321737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3399e4-ece8-4680-929c-031b6bc7a8dc None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.365169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.365709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.365987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.366634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.366901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:30.367201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.368632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.382306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 291/579] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:30.383152 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.404894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93a675a-a002-4e4c-babf-2ef809fe6084 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.419994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.447264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.447993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.448220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.448541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.456644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.458140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:30.470630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.471640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 289/580] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.477511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.508230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34807da9-817b-4b26-8138-151a8e9c2ff0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.547248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.547858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.548137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.548463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.549335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:30.549631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.550999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.554676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.567799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 292/581] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:30.567799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.592571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebe7afa-3691-4f02-8b92-29720bdde34a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.632259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.633212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.633681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.634123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.634938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:30.635460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.637130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.651008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 290/582] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:30.652183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.674101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.680871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba19a06-09fc-4050-8b6c-be6a39999856 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.716665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.717390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.717666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.717936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] PATH_INFO: `/v3/users/glance` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.726209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.727641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:30.738905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.739890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 293/583] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.740826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.774575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf2d9fd-b870-46a0-9da2-75168006c440 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.806659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.812327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.812938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.813270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.813569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.814552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:30.814857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.816357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.834007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 291/584] 10.222.0.73 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:30.834845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.858150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39d84e-0bad-4c83-9998-1dbbbed3750b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.896537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.897116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.897398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.897670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.910155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.911614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:30.927709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:30.928826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:30.934405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:30.934405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:30.934405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:30.934405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 294/585] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:30.934405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:30.942995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:30.943948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524240e-e431-4646-82c8-b9fa06de4cc6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:30.979918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:30.980912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:30.981286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:30.981635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:30.989555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:30.991014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:31.002489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.003385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:31.004281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 292/586] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.004768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.043866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc1fd1-0d6e-42bf-8024-af369ade167b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.059219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.081338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.082378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.082861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.083330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.084291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.084688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.086159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.108584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 295/587] 10.222.0.73 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.113957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.114684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021b758e-0b89-483a-8f60-dc9926e0e090 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.152823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.153418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.153699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.153976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.167286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.168766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:31.184959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.185952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:31.186829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:31.186829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:31.186829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:31.186829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 293/588] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.186829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.191901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.235093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebbb496b-201b-4374-b613-6c2b4a6f835f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.273679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.274408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.274691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.274968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.275629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.275940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.277412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.291792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 296/589] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.297130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.301630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d533a39-6096-4fc7-93e8-42ff6a9511cf None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.303236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.336453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.337075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.337354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.337623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.338313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.338757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.340179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.353868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 294/590] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.354783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.406460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc772c0d-7d8c-4de6-89c7-2123a5c6cc4c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.433321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.445949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.446797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.447126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.447475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.458806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.459160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c3dba0-a724-4dc6-88af-1038d1e2fb7d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.460243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:31.476174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.477183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:31.478115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:31.478115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:31.478115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:31.487306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 297/591] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.488033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.502455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.503067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.503344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.503611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.504255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.504556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.506024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.518977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 295/592] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.524176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.551159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.604707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac3961ea-e409-4543-a414-7c144d6ecc95 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.631482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff72daf-0f6c-43be-8499-b5831ae1922b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.644388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.645042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.645348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.645612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.653549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.654987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:31.667030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.668158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 298/593] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.669590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.671872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.672708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.674821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.675103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.675844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=a54377901631412abe887223a71d3623) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.676841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.677602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.693411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 296/594] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/role_assignments?user.id=a54377901631412abe887223a71d3623&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.693411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.730558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.813492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc27e50-1820-4505-b0fd-e7e380bafe19 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.830638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:31.851916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f70b6c-85dd-43da-80bb-e26bd924b1af None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:31.856898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.857737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.858157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.858565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.859436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:31.860563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.862665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.874435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 299/595] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:31.875335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:31.894663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:31.895295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:31.895568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:31.895834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:31.905582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:31.908104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:31.924933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:31.926004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:31.926900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:31.930093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 297/596] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:31.931172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.003377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.011807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892e342d-2064-4176-af40-7ccaaca49c06 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.047087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18c9524-dfcb-4ccc-823a-36155b4e7206 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.061653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.063105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.063898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.064860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.066249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.067144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.068823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.085196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.085914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.086191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.086461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.095357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 300/597] 10.222.0.73 () {56 vars in 1139 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.095564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.096710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.098663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:32.122527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:32.123482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:32.124442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 298/598] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:32.125645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.141662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.225410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1424402-f8b0-4233-936a-527e5b62a8d3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.253446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5631aac0-0170-4045-a175-3a0a6ebc87c0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.262013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.262623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.262899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.263166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.263849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.264146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.265589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.277912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 301/599] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.283484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.294098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.294737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.295015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.295288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.300002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.306459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.307974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-041b8655-39e9-4823-b848-c513c3177d28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:32.322373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:32.323332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:32.324241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:32.324241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:32.324241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:32.328215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 299/600] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:32.330042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252831b3-3c2d-4950-bee7-0b81fca76f6f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.330369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252831b3-3c2d-4950-bee7-0b81fca76f6f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.330673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252831b3-3c2d-4950-bee7-0b81fca76f6f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.332141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 300/601] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:32.336684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252831b3-3c2d-4950-bee7-0b81fca76f6f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.393516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc6e2a5-3d89-4e85-8d5d-532d0631c832 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.404529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.433901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.434537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.434816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.435086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.436200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.436512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.438408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.450507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252831b3-3c2d-4950-bee7-0b81fca76f6f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.457948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 302/602] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.458652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.492204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e040f-7ed4-4260-9187-fcf4ef84d63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.492911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e040f-7ed4-4260-9187-fcf4ef84d63a None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.493254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e040f-7ed4-4260-9187-fcf4ef84d63a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.493592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e040f-7ed4-4260-9187-fcf4ef84d63a None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/f17bae22fe2b4aa09619bdc0b3e0b4a3/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.512889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e040f-7ed4-4260-9187-fcf4ef84d63a None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=f17bae22fe2b4aa09619bdc0b3e0b4a3, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.514453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499e040f-7ed4-4260-9187-fcf4ef84d63a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.543223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.568811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124293f3-74cb-45db-8101-f1ecf999d110 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.592546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 301/603] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:32 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/f17bae22fe2b4aa09619bdc0b3e0b4a3/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:32.594125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.594410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.594686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.615288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.615958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.616880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.616880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.617461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:32.617819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.619395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.620625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:32.622239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:32.638124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 303/604] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:32.638124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.717446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 302/605] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:32.719045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:32.798372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.806409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54bfa6-66dd-4051-b162-b6a7556af9bd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.857733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.858832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.860155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.860689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/c498027e07ce4bc28bc0518f22047b16/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.865036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:32.883745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=c498027e07ce4bc28bc0518f22047b16, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.885577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.911576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af7e770-bd1b-4a2c-975d-9c4b55f04172 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:32.950932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:32.951825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:32.952247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:32.952659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:32.961213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:32.962915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:32.976136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:32.977246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:32.978417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:32.985512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 303/606] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:32.998146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 304/607] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:32 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/c498027e07ce4bc28bc0518f22047b16/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:32.998287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.023088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.148477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414ae5b7-4c2c-4fba-83af-f192c5e94267 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.192380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.193596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.194050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.194528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.196132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.205713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f94fc7c-cb53-48be-9477-e478f529b36c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.210170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:33.215577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:33.216583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:33.217472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:33.221094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 305/608] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:33.230415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.263877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.264506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.264836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.265093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.265808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.266136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.267630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.281880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 304/609] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:33.288814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.340538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.460069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc7495-b636-4039-8c0e-0104347c594b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.463637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff3ef60-1922-492b-9f4b-0b48f27b0e58 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.518362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.518985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.519551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.519824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.521081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.521392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.522804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.534998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 306/610] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:33.538950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.539620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.539892 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.540158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/b05fdc39b370474fa59a63b4b0c7748a/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.542838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.572738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=b05fdc39b370474fa59a63b4b0c7748a, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.574214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.671236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 305/611] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:33 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/b05fdc39b370474fa59a63b4b0c7748a/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:10:33.672698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7ad0fd-a079-4fb3-8f30-879859e49a0a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.689828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.717300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.718101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.718382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.718653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.732063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.733742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3caff790-9669-4998-87a5-9f1993b735cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:33.750020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:33.751068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:33.752142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:33.752142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:33.752142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:33.752142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 307/612] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:33.758303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:33.770990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:33.853170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68f07-8d81-45b3-aeb7-1a6dde19df99 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.853481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68f07-8d81-45b3-aeb7-1a6dde19df99 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.854089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb68f07-8d81-45b3-aeb7-1a6dde19df99 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.855941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 308/613] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:33.861725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.862115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.862463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.889445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:33.890845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:33.893646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40106d6a-0e3d-42df-a3f8-98d6f7855751 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:33.957253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:33.957928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:33.958207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:33.958477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:33.959842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:33.961036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:33.962418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:33.979213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 306/614] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:33.984829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.014400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 309/615] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:34.068822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.106547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360d2ef1-61cc-4558-9ca1-dbdb711cae17 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.148635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.149299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.149582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.149854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.157673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.159894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:34.171032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:34.171981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:34.173782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 307/616] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:34.178207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.320825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.333134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04039168-cd25-4821-9d9d-16d225ca63a2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.361659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.392754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.393437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.393895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.394173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.394964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:34.395277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.396733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.440203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 310/617] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:34.446995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.550999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a5fec5-1e03-428c-bdc8-493d72b2fe65 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.591028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3decacd2-024c-4074-bef4-4dda80e23ca8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.600616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.618707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.619350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.619629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.619898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.632611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:34.636063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.636714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.636997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.637298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:34.637771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:34.638669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:34.640070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 308/618] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:34.649990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.666907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:34.740248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.741774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.779203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87ac63-8c21-47e0-80d6-ab16a3ddaeda None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:34.822332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:34.823037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:34.823313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:34.823582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:34.824252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:34.824549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:34.825949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:34.846055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 309/619] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:34.851199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.950699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 311/620] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:34.956175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:34.957386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.032071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5643a6-36ac-4fca-a21b-8ae1bd9a83e3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.041411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.073438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.075013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.075507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.075986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.090769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.093164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.099608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144b80c2-490e-4ce2-a588-96d8b75eb51c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d08ad909-3063-448d-b2ac-2ac76dae54ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:35.115655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:35.116691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:35.118039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:35.118039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:35.118039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:35.118950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 310/621] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:35.119775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8642b-e8eb-4a62-82f9-d07e8aeefca4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.120057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8642b-e8eb-4a62-82f9-d07e8aeefca4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.120327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d8642b-e8eb-4a62-82f9-d07e8aeefca4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.127064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 311/622] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:35.128293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8642b-e8eb-4a62-82f9-d07e8aeefca4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.145864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.146940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.147425 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.147921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.160513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.162450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6d97a8d3-6dc8-480d-9e3b-5bef4ddec6ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:35.179139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:35.180146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:35.181050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:35.181050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:35.181050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:35.182729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 312/623] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:35.184582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.185159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.185641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.206593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:35.208182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:35.246007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d8642b-e8eb-4a62-82f9-d07e8aeefca4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.248294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.282242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 313/624] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:35.284571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.294857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.295478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.295752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.296019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.296772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:35.297099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.299363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.317396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.324683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 312/625] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:35.329891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.395475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c94dac-bcea-43ed-abd2-41b7807a66a1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.432120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.432774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.433085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.433372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.434038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:35.434349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.436134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.449564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 314/626] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:35.458127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.545548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f52365-ebbf-439d-a15b-61f20a91ab03 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.577934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.591968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ddc7c-6dd2-4282-bc15-963eb45a7541 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.612983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.614005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.614495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.614996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.623343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.624797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.632898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.633902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.634408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.634886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c15614ca-1c1d-4af6-be23-c1b9a42dba3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:35.637002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:35.637938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:35.639241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 313/627] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/users/swift => generated 83 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:35.639836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eed9b-7014-428d-97d2-0987699daa2a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.639906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eed9b-7014-428d-97d2-0987699daa2a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.640230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816eed9b-7014-428d-97d2-0987699daa2a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.646289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.650524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 314/628] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:35 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:35.650524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eed9b-7014-428d-97d2-0987699daa2a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.650746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-022e98a4-7996-4372-b728-55f128b8652a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:35.665880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:35.666943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:35.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 315/629] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:35.671081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.671438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.671783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.687892 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.692039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:35.693353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:35.770404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 316/630] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:35.771389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.792158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816eed9b-7014-428d-97d2-0987699daa2a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.843913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.846319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.846859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.847267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.848419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:35.848922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.850885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.874054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 315/631] 10.222.0.73 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:35.875179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.888707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.920910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5151ce54-95cb-48dd-ab17-d7049edd763e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:35.971216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:35.972081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:35.972612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:35.973545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:35.973692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:35.975237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:35.976168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:35.978084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:35.994475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 317/632] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:35.996353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:35.998202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37263f3d-34c1-4835-9fb1-1b77a1543b01 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.047533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.049511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.049905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.050341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.068259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:36.077293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:36.078424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:36.079626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:36.080395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 316/633] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:36.081215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.146278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd96fd-4844-4242-bd60-55749b6de448 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.195196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafd43c4-34aa-4a5e-af52-0a73e48eb825 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.203071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.204160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.204698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.205272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.218132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.224618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.227211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.227689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.228298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.228577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.228846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.229614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=059f72c1793e492fb052bebbf29b5bec) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:36.229940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.231351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.239060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 317/634] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/role_assignments?user.id=059f72c1793e492fb052bebbf29b5bec&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:36.240399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:36.243619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:36.244653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:36.245580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:36.245580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:36.245580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:36.246458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.249709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 318/635] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:36.250675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.381954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9f15c-4778-4ffd-afeb-4f2fd332736e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.400000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66715b43-6c57-42f1-99e1-29f4b865bbf6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.429612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06903bf-4abd-4043-87a8-b926adf30436 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.430400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06903bf-4abd-4043-87a8-b926adf30436 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.430768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06903bf-4abd-4043-87a8-b926adf30436 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.431039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06903bf-4abd-4043-87a8-b926adf30436 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.444418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.445057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.445489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.445799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.446479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:36.446596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06903bf-4abd-4043-87a8-b926adf30436 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:36.446789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.448555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d06903bf-4abd-4043-87a8-b926adf30436 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:36.452002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:36.453141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:36.454528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 318/636] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:36.456827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8518363-13fa-4754-8565-1a6c2fa10512 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.456827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8518363-13fa-4754-8565-1a6c2fa10512 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.457129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8518363-13fa-4754-8565-1a6c2fa10512 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.460749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 319/637] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:36.460749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8518363-13fa-4754-8565-1a6c2fa10512 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.465779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 319/638] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:36.466515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.532682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.544455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.593277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3843e3-37fc-4eb4-8e6b-7776a0afe910 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.593966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8518363-13fa-4754-8565-1a6c2fa10512 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.631948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.632576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.632856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.633163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.633843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:36.634143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.635524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06b7b60-82ff-44a1-9960-4017a09e8ce2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.637308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.637927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.638211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.638486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.639122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:36.639422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.640785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.651900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 320/639] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:36.651900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.651900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.651900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.654647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 320/640] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:36.660109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.673961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:36.675475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:36.758953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 321/641] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:36.760081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.789970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40ed93-15f5-4cb2-af90-2e3ceb153b03 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.814479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.826209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:36.839845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.840494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.840777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.841067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.861052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.861052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:36.881731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:36.883673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:36.885412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:36.885412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:36.885412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:36.889414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 321/642] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:36.890600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:36.896561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0513d180-2d51-4c4e-a222-7b2c0028dae1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:36.941656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:36.942335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:36.942678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:36.942874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:36.954510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:36.956284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:36.973850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:36.974951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:36.975829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:36.975829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:36.975829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:36.979987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 322/643] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:36.981092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.029353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-753f456e-ea69-4cff-b098-2fdeb32fd278 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.078117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.079099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.079099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.079421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.086480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.086833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.088236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.092479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc643b26-725b-4dba-b13e-57dfa3956687 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:37.101255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:37.106545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 322/644] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:37.107492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.138935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.140035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.140847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.141460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.142371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:37.142892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.144647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.145322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.161731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 323/645] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:37.165213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.222410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb67b9c-a8dd-489f-991d-e1a507cba02b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.260464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.261098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.261376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.261643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.262286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:37.262586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.263935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.276029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e177fb-5a39-4117-abfb-19c84c007e81 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.281292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 323/646] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:37.281292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.317111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.317755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.318100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.318304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.329180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:37.334467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:37.335411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:37.336225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:37.340614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 324/647] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:37.341572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.360926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.383084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.389431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df688fc-b6f4-43db-ba59-98f605e30a5e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.428286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.429050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.429448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.429720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.430410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:37.430710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.432107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.445094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 324/648] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:37.445927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.459500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461d1521-1187-4762-b0c7-24e4d39b6530 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.498958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.499592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.500197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.500478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.508975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.510479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:37.523230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:37.524174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 325/649] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:37.526386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.566381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d39e9b-833a-41f6-98b2-6d663fa90571 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.600212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.600861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.601170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.601443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.608884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.610325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.614617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:37.622175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:37.623147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 23 22:10:37.624080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:37.628489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 325/650] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:37.637727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.642584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21d044b-caf6-44ec-96a6-c8ec96cb038b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.648084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.678464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.679064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.679341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.679610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.680265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:37.680568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.682057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.692926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 326/651] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:37.693699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.749982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2b48d9-48a4-49d4-aecc-c4ba57ad1866 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.787487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.788172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.788420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.788695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/a54377901631412abe887223a71d3623/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.804419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=a54377901631412abe887223a71d3623, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.805947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.807537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e509a0de-00c3-4368-9cc0-d76f69b95864 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:37.841476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:37.842119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:37.842345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:37.842616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:37.843343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:37.843649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:37.845689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:37.854685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:37.864736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 327/652] 10.222.0.73 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:37.865584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.883440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 326/653] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:37 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/a54377901631412abe887223a71d3623/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:37.884143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:37.965208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.009861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a215c-ed9c-44ec-ad49-ff343be086fc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.026273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab00b49d-9a7a-4ea6-8c98-a272400824d3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.056464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.057122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.057409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.057686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.071077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.072579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.074111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.074727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.075288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.075568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.076747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:38.077081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.078503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:38.090589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:38.091594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:38.092501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:38.092501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.092501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:38.099196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 327/654] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:38.099196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.103539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 328/655] 10.222.0.73 () {56 vars in 1137 bytes} [Tue Aug 23 22:10:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:38.105548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.187696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.240148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a53e52-216b-4757-b41c-2e4ac5bcb14e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.254026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815a149-e372-4112-b2ae-9461327e9a54 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.266313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.290937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.292365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.293216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.293712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.302118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.302740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.303024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.303301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.303961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:38.304267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.305686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.326396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 329/656] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:38.326396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.385897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.388194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.439140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826c6491-b4b7-457f-9f6d-64e3b644a76f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.453888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.477850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde1389f-4a20-4101-a9a2-c7eb0ec90f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.478566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1389f-4a20-4101-a9a2-c7eb0ec90f51 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.478937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1389f-4a20-4101-a9a2-c7eb0ec90f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.479303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde1389f-4a20-4101-a9a2-c7eb0ec90f51 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.510104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 328/657] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:38.510968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.530987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.565176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1389f-4a20-4101-a9a2-c7eb0ec90f51 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.566831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde1389f-4a20-4101-a9a2-c7eb0ec90f51 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.627157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3154456a-f273-4a84-b2ce-7bf90e6b9894 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.666506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.667138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.667420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.667707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.675433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 330/658] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:38.675622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.676825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.681124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a1a8-e29e-47f4-b840-ed6b90ce7305 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.681410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a1a8-e29e-47f4-b840-ed6b90ce7305 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.681679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e298a1a8-e29e-47f4-b840-ed6b90ce7305 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.683156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 331/659] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:38.683673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298a1a8-e29e-47f4-b840-ed6b90ce7305 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:38.689854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:38.690863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 329/660] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:38.696749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.710268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.795888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.808320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e298a1a8-e29e-47f4-b840-ed6b90ce7305 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.821575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bb111b-016b-4d60-8713-256f85dc9151 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:38.845210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.846496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.846950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.847532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.861189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.861905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:38.862661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.862850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.863074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.863319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.874497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:38.875966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-60fb8f0f-61c7-4f89-9e68-f4da5795a615 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:38.879184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:38.880126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:38.881179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:38.881179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.881179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:38.881179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 332/661] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:38.882361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:38.882361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:38.882361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:38.899492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:38.901277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:38.902185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:38.902185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:38.902185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:38.906094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:38.906864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 330/662] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:38.907998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:38.908828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:38.974807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:38.985091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 333/663] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:38.985091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.045501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131172f3-d3da-4a40-90d0-d6fa6d632e62 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.084895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.094714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948f691-69b3-4c8e-a40a-0848d35f0151 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.103250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.103250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.103250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.103250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.103635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:39.103635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.103635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.121726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 331/664] 10.222.0.73 () {56 vars in 1139 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:39.122585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.145404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.146237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.146660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.147621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.148415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:39.149148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.151334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.167527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 334/665] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:39.168612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.228951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.240535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-384997f3-b68a-43b1-88ce-dfff94e20908 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.279927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.280693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.281304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.281577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.282261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:39.282570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.284673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.285196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83d5763-583a-41ba-8b65-0a84244bc491 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.296828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 332/666] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:39.298540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.326871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.327522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.327767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.328546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.359977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.409464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.410885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.426832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad73f9f-4f05-47fd-a06a-b343d6115224 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.466571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.467215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.467494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.467990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.475219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.476509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.477320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:39.488764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:39.492866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:39.493810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:39.494910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 333/667] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:39.495615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.541707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 335/668] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:39.541707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.615283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53e1d38-b0b0-41ed-9403-e6d4ff6d66c9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.627726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.658505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.659560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.660290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.660512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ae501e-8128-4cea-8740-3dc3340260aa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.661222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.674500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:39.679902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:39.680933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:39.681827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 334/669] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:39.682540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.716326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.717943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.718330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.718615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.727348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.728821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:39.741699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:39.742719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:39.743627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 336/670] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:39.744845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.769819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.805287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e532aea1-ed87-4443-a633-add8624a387e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.844698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.845711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.846026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.846303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.847090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:39.847298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.849055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.869443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 335/671] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:39.869443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.874315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bff057-d1d9-4228-a19d-cb36b0e72d03 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:39.899040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:39.920935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:39.920935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:39.920935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:39.920935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:39.924220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:39.925723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:39.940472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:39.941829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:39.942921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:39.942921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:39.942921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:39.942921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 337/672] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:39.942921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:39.978442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74d5043-2ed1-45de-acfe-0904bd65b929 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.010734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.014461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.015143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.015422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.015693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.016364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.016747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.018194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.026546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 336/673] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.027375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.046292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e881610-99c0-426e-950b-9b77dba8ff55 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.081096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.081705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.081977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.082242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.082924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.083221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.084664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1371d7d-091c-44ce-b39b-6fefc2f5995b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.097312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 338/674] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.098801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855ad43-8f74-4040-be67-b176395fd4f6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.099092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855ad43-8f74-4040-be67-b176395fd4f6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.099394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855ad43-8f74-4040-be67-b176395fd4f6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.101080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 339/675] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:40.101335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855ad43-8f74-4040-be67-b176395fd4f6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.127710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.138638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06b484-aa33-4ce3-b1f5-b283eae2a38f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.177064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.177589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.178652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.178934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.179656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=f17bae22fe2b4aa09619bdc0b3e0b4a3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.179957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.181403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.187343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 337/676] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/role_assignments?user.id=f17bae22fe2b4aa09619bdc0b3e0b4a3&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.188131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.211636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855ad43-8f74-4040-be67-b176395fd4f6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.244555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.254182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.254817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.255099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.255374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.256049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.256354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.257801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.271322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 340/677] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.272266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.311191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2141ab3-af01-48f4-8fc3-5e7d2cadf33c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.353102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.353747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.354029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.354305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.367331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.368774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b3a259fd-f701-4e7f-a44d-be3cc1b18971 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:40.385625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:40.386624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 338/678] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.388542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.397933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fd1bd9-3747-42f7-afb8-d097b6ee6a77 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.401654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.408104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:40.409769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:40.435734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.436715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.437000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.437302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.438031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=c498027e07ce4bc28bc0518f22047b16) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.438340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.439842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.445854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 341/679] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/role_assignments?user.id=c498027e07ce4bc28bc0518f22047b16&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.446630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.521073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 339/680] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:40.521073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.526614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.622786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38512978-503f-45fa-9ab0-d2d148634bf9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.681024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.681790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.682134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.682477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.693388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.694869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.695842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d894b7f-1a64-411b-a0bd-4f2f2adebf56 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:40.712874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:40.713877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:40.714776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:40.716674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 342/681] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:40.719413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.738805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.739744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.740233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.740721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.741609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.742155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.743733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.746836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.758186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 340/682] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.763860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.843640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99e80b-8298-414a-86c5-f878a658b2e4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.862314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:40.884819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.886272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.886758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.887243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.888181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:40.888707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.890380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.895222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a365338-a0f9-4072-b09b-3623c112ff48 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:40.908729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 343/683] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:40.909738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:40.939199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:40.939838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:40.940120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:40.940395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:40.949494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:40.950945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:40.963328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:40.964323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:40.965292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 341/684] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:40.965904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.021611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1090cda2-a20a-4b1d-9b7d-133561916849 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.025751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.060983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.061892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.062351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.062802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.074187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f573f6fe-93ed-4dce-ad37-1b395fd86886 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.075912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:41.080951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:41.082172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:41.083108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:41.085347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 344/685] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:41.086307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.116287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.116914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.117243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.117516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.118372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=b05fdc39b370474fa59a63b4b0c7748a) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:41.118597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.120022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.122759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.133207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 342/686] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:40 2022] GET /identity/v3/role_assignments?user.id=b05fdc39b370474fa59a63b4b0c7748a&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:41.135894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.245599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1686d797-d2c0-48cc-9076-ce537286e71f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.276648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482607d8-b103-46fe-a1d1-66e49dee202a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.300538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.318202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.318828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.319110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.319355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.319481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.320052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:41.320221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.320358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.320691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.321089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.321819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.322176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:41.322579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.324868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.332948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 343/687] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:41.340622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.371107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 345/688] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:41.378095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.504726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d4227-4853-4191-ba64-76df9edf83c4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.515200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.529639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d7f2f-d59c-42cf-8540-824fbd9ee9fe None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.548394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.549197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.549484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.549859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.568326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.570320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.574850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.575464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.575745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.576018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:41.588745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:41.589960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:41.590853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:41.590853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:41.590853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:41.593122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 344/689] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:41.604260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.646195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.668488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.669841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.730296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcb3fc4-55a9-4089-992d-5329b2a4c291 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.775339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.776425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.776963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.777663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.778610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:41.779124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.780836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.784477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.801095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 346/691] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:41.801301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 345/691] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:41.802544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.805196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:41.905980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:41.913534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05394fe2-8c90-45e3-9e86-53c1a38b0fe7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.918145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a4d1db-b8eb-43af-abff-84fff7717f9f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:41.958923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.960013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.966888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.966888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.974550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:41.976416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:41.994519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:41.995558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:41.996528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:41.996528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:41.996528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:41.997900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:41.998567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:41.998849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:41.999117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:41.999811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 347/692] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:42.006317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.018335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.020063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:42.047207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:42.050008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:42.051190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 346/693] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:41 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:42.054857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.064492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.152549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b007502-4eba-405b-ab67-533c44f280b3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.203230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.203854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.204864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.205191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.205851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:42.206153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.211208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.241942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 348/694] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:42.243125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.277150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00484373-801a-4f7e-ac87-e0d86788e2a9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.354691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.355244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.355521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.355790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.356538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:42.356837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.358624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.396769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 347/695] 10.222.0.73 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:42.402744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.421392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.471299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b6668e-2cdf-4b54-89ca-1977ef0f628d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.473961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.512731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.513417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.513701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.514547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.526177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.528023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.529118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c822bab-1475-4e42-93c0-822712232830 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-143529b3-b860-43fa-ac30-03d59a8fa300 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:42.546113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:42.547081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:42.548028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:42.549182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 349/696] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:42.550655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999f09-db86-40db-8f9c-5416dc554946 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.551047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999f09-db86-40db-8f9c-5416dc554946 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.551424 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f999f09-db86-40db-8f9c-5416dc554946 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.554698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 350/697] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:42.555306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f999f09-db86-40db-8f9c-5416dc554946 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.582065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.582962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.584490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.584865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.700002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.702423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.714177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f999f09-db86-40db-8f9c-5416dc554946 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.720765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.762067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:42.763010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.763010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.763210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.763919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:42.764221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:42.765710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f908c4dc-ea85-4c98-aaa2-d02403d6c5e9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:42.785135 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 351/698] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:42.786433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:42.786724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:42.787002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:42.808710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:42.810687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:42.831789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 348/699] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 23 22:10:42.832561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:42.834338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.901432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 352/700] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:42.902903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:42.978169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97022954-78a9-41f9-b5f0-d2e8d0f22899 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:42.996558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.022073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.022465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58278f7-7617-430b-a269-e55d1f852e19 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.022915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.023011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.023222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.024049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=059f72c1793e492fb052bebbf29b5bec) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:43.024364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.025955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc08b64-db90-42a8-8c01-a3378652963a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.032888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 349/701] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/role_assignments?user.id=059f72c1793e492fb052bebbf29b5bec&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:43.036678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9c5c-3ed6-4c85-9109-67be2d4a40fe None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.036678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9c5c-3ed6-4c85-9109-67be2d4a40fe None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.036678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993c9c5c-3ed6-4c85-9109-67be2d4a40fe None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.041479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 350/702] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:43.044428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.044734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.044998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.071943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:43.073431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:43.095094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.096641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.097171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.097703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.120817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.121564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:43.154193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:43.155292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:43.156337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:43.156337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:43.156337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:43.161879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 353/703] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:43.162724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.172141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 351/704] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:43.191044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.334219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.351364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c3c6e7-f863-48a2-ae9e-db26f4da5475 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.401844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.402768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.403211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.403634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.404504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:43.404968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.406598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.432834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 354/705] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:43.442086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.526405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed22af9-e682-4c8f-924c-14cdceb48d5e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.536191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.545543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.575911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.576856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.577303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.582713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.582713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.582713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:43.592514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:43.593747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:43.595317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:43.598018 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 352/706] 10.222.0.73 () {56 vars in 1129 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:43.599149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.615834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9de3bb4-dd88-4b95-9c8f-a2d2ee2ed94c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.667547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.668368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.668741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.669138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.677502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.679111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:43.691806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:43.692998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:43.693995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:43.694791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 355/707] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:43.696182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.757156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6a1d6-b04a-4f1f-87f4-576ea8a77454 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.813104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.815320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.815948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.816225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.818506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249b370b-c5d2-4842-887a-f74725b3af71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:43.834378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.836047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:43.852860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:43.853876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:43.854762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:43.854762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:43.854762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:43.854762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 353/708] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:43.855435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.856057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.876476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:43.877226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.877503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.877773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.878403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:43.878703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:43.880116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f761bd5c-0258-49cf-bde8-7a03c45f78d1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:43.889089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 356/709] 10.222.0.73 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:43.889089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b73c-e642-4324-baab-ad4435fae8d1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:43.889089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b73c-e642-4324-baab-ad4435fae8d1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:43.889089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d354b73c-e642-4324-baab-ad4435fae8d1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:43.889089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 357/710] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:43.889089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b73c-e642-4324-baab-ad4435fae8d1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:43.897558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:43.999769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a5710-ce11-4834-874b-04b901d9ecba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.006829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d354b73c-e642-4324-baab-ad4435fae8d1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.044296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.045407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.045987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.046547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.047501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.048437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.049664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.050130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.050315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.050578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.050848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.051665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.051967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.053818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd176423-a8f6-4412-86ff-8027b4733fb0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.069090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 354/711] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.069090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.069507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 358/712] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.069507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.069507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.069842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.090277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:44.092441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:44.141898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.164278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.167592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 359/713] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:44.168527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.202959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbfdbb2-5852-46e9-803e-1dbe6f1649f2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.247280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.248070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.248417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.248784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.249591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.250007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.252009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.260581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 355/714] 10.222.0.73 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.262059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.304039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ad5411-4917-4f03-a89e-fe701b4e06a0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.386913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.387544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.387845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.388098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.390046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=a54377901631412abe887223a71d3623) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.390382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.392628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.409356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 360/715] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/role_assignments?user.id=a54377901631412abe887223a71d3623&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.456020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0cf8b9-5354-4730-91b8-d112a5799977 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.488475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.517371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.517999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.518306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.518646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.519300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.521452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.548608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.609571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 356/716] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:10:44 2022] POST /identity/v3/users => generated 254 bytes in 348 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:44.638092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500ddab8-456e-47ce-99d6-f451529f39b8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.684443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.685648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.686157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.686453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.687047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.688994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:44.708567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:44.709665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:44.710546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:44.713765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 361/717] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/services/network => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:44.720329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:44.723986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:44.846028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b6cafb-c759-49cf-831a-860fbef21d18 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:44.883037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:44.883643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:44.883922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:44.884244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:44.884880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] RBAC: Adding query filter params (name=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:44.885235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:44.886652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5d35ce-ddc4-41ff-977c-98e00bcbe754 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:44.892554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 357/718] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:44.901104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:45.015792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18918b0-43d5-4449-8c5e-4db5ef997727 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:45.024907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:45.057109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:45.057771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.058008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.058340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.059564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:45.060375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:45.061813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6c551e-edd4-4c9f-a6fd-9f79f0de3dad None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:45.069406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 362/719] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:45.374990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f964f7-4b38-47b3-9cdf-ba7f36ec407e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.375481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f964f7-4b38-47b3-9cdf-ba7f36ec407e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.375481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f964f7-4b38-47b3-9cdf-ba7f36ec407e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.378088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 358/720] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:45.384560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:45.384843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:45.385148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:45.408306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:45.409813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:45.509807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 363/721] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:45.887581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f964f7-4b38-47b3-9cdf-ba7f36ec407e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:45.989322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.037413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f964f7-4b38-47b3-9cdf-ba7f36ec407e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.081301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.081944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.082223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.082496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.097524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:46.102750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:46.103691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:46.104934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 359/722] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:46.110082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.243071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.246629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7334194c-171f-419d-bbbc-041311d7c9e3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.296643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.297317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.298234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.298641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.299522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:46.299831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.301286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.317759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.320802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 364/723] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:46.468246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6317e5b-0550-43ba-b0db-b18ed99f1d6b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.505537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.511379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.512022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.512344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.512632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.525445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.526930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:46.543047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:46.544057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:46.545556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:46.545556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:46.545556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:46.545556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 360/724] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:46.550397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.726583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0079cb-2240-43d5-b124-d33b055ea18e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.777909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:46.778820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.779283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.779733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.780562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:46.781080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:46.783898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3e1161-273a-4f1c-bba5-696c4c40332f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:46.801186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 365/725] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:46.806430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:46.814922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15cd51-17e3-48f6-8935-306ef3b04f0f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.815498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15cd51-17e3-48f6-8935-306ef3b04f0f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.815954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa15cd51-17e3-48f6-8935-306ef3b04f0f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.817769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 366/726] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:10:46.823355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49824-83db-4672-82d8-62759299303d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.823857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49824-83db-4672-82d8-62759299303d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.824314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa49824-83db-4672-82d8-62759299303d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.854748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3aa49824-83db-4672-82d8-62759299303d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:46.857547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3aa49824-83db-4672-82d8-62759299303d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:46.898449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:46.977913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817cc94-711b-4940-bdcd-3f42333b0d93 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:46.978776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 367/727] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 23 22:10:46.980381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9a31-d62b-4046-98f1-531283d33f02 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.980984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9a31-d62b-4046-98f1-531283d33f02 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.981453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8e9a31-d62b-4046-98f1-531283d33f02 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:46.987506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 368/728] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:46.989095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:46.989590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:46.990014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.009426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:47.010873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:47.049955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.052041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.052897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.053735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.069435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.072044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:47.093045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:47.094371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:47.095724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:47.097968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 361/729] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:47.107125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.111314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 369/730] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:47.173325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.218395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e74fd-5b11-4e9b-a072-a1885f2218fe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.260267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.260890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.261199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.261474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.262215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:47.262523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.263966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.291460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 362/731] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:47.296783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.371200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.438680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.460540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2dd86ba-8dca-42dd-816b-8c8e23d64856 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.502471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.503472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.503986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.504458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.512125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.517946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.519653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.519872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8139ae1d-e2c0-4fe6-9468-93112a9fb3f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-986886e8-39b0-442c-8c81-adf9a6da926a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:47.537065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:47.538070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:47.539680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:47.539680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:47.539680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:47.539680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 370/732] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:47.539680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febb0f2-7959-4e50-8284-052922080867 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.539680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febb0f2-7959-4e50-8284-052922080867 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.540056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5febb0f2-7959-4e50-8284-052922080867 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.545271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 371/733] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:47.546094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febb0f2-7959-4e50-8284-052922080867 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.563953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.564912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.565128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.565355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.566016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.567519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.643679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 363/734] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:47 2022] POST /identity/v3/services => generated 251 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:10:47.644509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.660768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5febb0f2-7959-4e50-8284-052922080867 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.696710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.706770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67efa08-9984-4de0-89c7-6e151b6eacf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.708398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67efa08-9984-4de0-89c7-6e151b6eacf5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.708828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67efa08-9984-4de0-89c7-6e151b6eacf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.709152 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67efa08-9984-4de0-89c7-6e151b6eacf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.722676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67efa08-9984-4de0-89c7-6e151b6eacf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c67efa08-9984-4de0-89c7-6e151b6eacf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:47.727733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:47.728776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:47.729699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 372/735] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:47.730946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7ef3b-2c06-46ee-8202-8e1846d17c5c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.730946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7ef3b-2c06-46ee-8202-8e1846d17c5c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.731107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7ef3b-2c06-46ee-8202-8e1846d17c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.750268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fee7ef3b-2c06-46ee-8202-8e1846d17c5c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:47.752371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fee7ef3b-2c06-46ee-8202-8e1846d17c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:47.778404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae54901-920d-4145-a597-a8183b551b61 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.822281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:47.822691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.822979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.823318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.823975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:47.824812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:47.825782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13823b40-e7a1-4bfd-8499-9d3893c3077e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:47.835035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 373/736] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:47.836977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6ed5e-27aa-4077-a922-d28e03140ae1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.836977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6ed5e-27aa-4077-a922-d28e03140ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.836977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d6ed5e-27aa-4077-a922-d28e03140ae1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.838400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 374/737] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:47.842673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6ed5e-27aa-4077-a922-d28e03140ae1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.845828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 364/738] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:47.847233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:47.847665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:47.848187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:47.869528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:47.871052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:47.883947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:47.952155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 365/739] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:47 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 23 22:10:47.962376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:47.966128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d6ed5e-27aa-4077-a922-d28e03140ae1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:47.997557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.025298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.026301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.026824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.027288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.028172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:48.028670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.034706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.053594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 375/740] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:48.059653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.084596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962b1a8c-073f-488d-a536-adb1cfc0625a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.120990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.121665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.122177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.122284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.129638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.131108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:48.145465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:48.146423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:48.147289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 366/741] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:48.150896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.160642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.227584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f48d364-4a85-4d68-a3b9-49b75d124448 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.267812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.268507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.268793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.269102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.284083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.285906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.302269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1548985b-9676-40ab-af13-8e2cf26ed126 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:48.306077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:48.307040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:48.307923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:48.307923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:48.307923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:48.307923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 376/742] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:48.309800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.346654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.347890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.348183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.348456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.349432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:48.349967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.352037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.353957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.367812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 367/743] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:48.373169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.430316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0d706-cff0-44f8-99dc-df5f03a2b84c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.445518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.486367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb216ed8-b11a-465a-9ff0-fbc2731f2887 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.523871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.524548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.524822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.525064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.525728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:48.526034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.527473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.545617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 368/744] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:48.545617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.585620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.586233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.586510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.586782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.587428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.588795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.657718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6a2779-a264-4efa-b3a9-1ebcfa9e3293 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.685786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.690967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 377/745] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:10:48 2022] POST /identity/v3/services => generated 240 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:48.691791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.719073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.720203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.720732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.721255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.735539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:48.741271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:48.742326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:48.743224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:48.746056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 369/746] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:48.747393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:48.854339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:48.864554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddf5965-aa4f-4fda-aa85-06c514dbe2de None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.893272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843a3d8-2dd3-4c57-8410-7f45eeb2546b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:48.914736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.915426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.915703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.916069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/059f72c1793e492fb052bebbf29b5bec/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.919290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=059f72c1793e492fb052bebbf29b5bec, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.921547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.942039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:48.942804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:48.943090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:48.943356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] PATH_INFO: `/v3/users/swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:48.952573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:48.954722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:48.967933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:48.969480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:48.970719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:48.971876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 370/747] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:48.973226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.008489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 378/748] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:48 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/059f72c1793e492fb052bebbf29b5bec/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:49.009672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.054327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:49.178714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4eb3d6-4f49-4898-89be-9aeaa02e022b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.210911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1a1372-0ef8-4e8c-b00a-69dd00071efa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.247161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.248978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.249583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.250384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.251869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:49.252664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:49.256209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:49.263891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.264531 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.264835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.265262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.268267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:49.269066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:49.271702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 371/749] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:49.271919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:49.279748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.308441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 379/750] 10.222.0.73 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:49.313104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.331864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:49.403909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedcfded-c8ee-44dd-ae8f-b428ab14f801 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.409409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:49.429252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f086ba6c-0d30-4cc2-b34e-e52a3b22032d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.444497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.445159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.445450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.445753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.457707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:49.459213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:49.468908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f0f31-3b67-4ada-a110-ecafbba9b6f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.469918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0f31-3b67-4ada-a110-ecafbba9b6f1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.470507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0f31-3b67-4ada-a110-ecafbba9b6f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.470988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f0f31-3b67-4ada-a110-ecafbba9b6f1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:49.475694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:49.476933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:49.478110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:49.478110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:49.478110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:49.478110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 372/751] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:49.481895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.590160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f0f31-3b67-4ada-a110-ecafbba9b6f1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:49.592126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f0f31-3b67-4ada-a110-ecafbba9b6f1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:49.597106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b863e9a7-1bff-4693-8129-5bc4b40e0987 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.621386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:49.640905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.642022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.648227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.648227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.648227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:49.648227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:49.648227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:49.666461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 373/752] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:49.673443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.719138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:49.827204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 380/753] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:49.828423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dfae-7b92-4bef-84e1-73b063b7141d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.828746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dfae-7b92-4bef-84e1-73b063b7141d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.829107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed0dfae-7b92-4bef-84e1-73b063b7141d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.831063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 381/754] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:49.836102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0dfae-7b92-4bef-84e1-73b063b7141d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:49.852234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238a074e-cbad-408b-8b6b-d7ff1e15916c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.890236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:49.891586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.892121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.892590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.900629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:49.902414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3856246e-00e9-4af7-9afe-0588e19e69cd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:49.917154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:49.918161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:49.919115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:49.921974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 374/755] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:49.924301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:49.924858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:49.925379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:49.946116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:49.947676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:49.965243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed0dfae-7b92-4bef-84e1-73b063b7141d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:49.984668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.014277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.016001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.018078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.018353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.031853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.033316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.040510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 375/756] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:50.040510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.055073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:50.060537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:50.061534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:50.062501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:50.062501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:50.062501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:50.064176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 382/757] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:50.069989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.184466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80d2fb-334e-4b64-9121-a43ff19129d8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.226040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.226658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.226941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.227268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.227920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:50.228223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.229782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.251068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 383/758] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:50.254359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.274092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acecefbe-5b51-4b4e-a962-b70a3a2b2114 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.306891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.330108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.331222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.331661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.332109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.332993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:50.333483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.335043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.370238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 376/759] 10.222.0.73 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:50.376285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.429526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e328c6-c8d4-43eb-9751-c4469a6d084d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.464461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.477079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.477737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.478017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.478287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.494247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.495724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:50.508477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:50.509460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:50.510342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:50.513131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 384/760] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:50.514172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.529913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b98bfb-92da-44f9-ba59-d3cf513ac993 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.580464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.580464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.580954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.581441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.628652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.648094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6481891a-518a-4bd8-abdc-fa3e821eb938 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.660233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.661770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.685839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.686771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.687251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.687710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.689274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.690818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:50.700090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:50.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:50.701976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:50.710634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 385/761] 10.222.0.73 () {56 vars in 1129 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:50.711788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.728799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.771863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 377/762] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:50.772992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.822351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13016162-a428-420a-b119-8298b21f81b5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.859453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.860132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.860428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.860697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.861419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:50.861724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.863164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.878715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3617a35-9d58-41b9-bc39-acc5207a95fe None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:50.880351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 386/763] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:50.881178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.892588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.916059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:50.916670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.916947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.917247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.917879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:50.918180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:50.919577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a311d75c-d345-44d4-8f30-bab97dda1575 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:50.924655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 378/764] 10.222.0.73 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:50.925878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef8a35-7eb9-47d4-b65d-8456849ed354 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:50.926161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef8a35-7eb9-47d4-b65d-8456849ed354 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:50.926429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ef8a35-7eb9-47d4-b65d-8456849ed354 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:50.929141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 379/765] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:50.929141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef8a35-7eb9-47d4-b65d-8456849ed354 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:50.958933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:50.998617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb923ff1-78a3-4ad5-a61f-b6548e07d596 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.037299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.037935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.038242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.038479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.039006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ef8a35-7eb9-47d4-b65d-8456849ed354 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.049878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.051465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:51.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:51.068038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:51.068926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:51.068926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:51.068926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:51.068926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 387/766] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:51.069172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.085150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.085914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.086248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.086586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.087375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=a54377901631412abe887223a71d3623) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.087764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.089284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd2757e-05a5-4e98-88ca-84fbba6730b5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.095402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 380/767] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:50 2022] GET /identity/v3/role_assignments?user.id=a54377901631412abe887223a71d3623&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.113058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.113058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.113058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.127710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:51.129215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:51.145421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.197265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1574b05-ce85-4cdc-ad2a-1d8c694fb4ee None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.226350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 381/768] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:51.227581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.258604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.269874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.270673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.271070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.271412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.272131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.272529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.274086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.287736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 388/769] 10.222.0.73 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.294763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.356135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b411776b-a67b-448f-8d65-96b51d2afe04 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.425840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.426469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.426746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.427011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.427719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.428022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.429402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.452432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 382/770] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.458362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.477888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df1f627-b151-4030-af3e-a45657e946f5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.504079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.520765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.520765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.520765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.520765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.521249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] RBAC: Adding query filter params (interface=public, service_id=58b08cc9af7a46149fb30715f069019c, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.521249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.521249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e434c0-3406-4180-8dfa-270df0073121 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.530046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 389/771] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/endpoints?service_id=58b08cc9af7a46149fb30715f069019c&interface=public®ion_id=RegionOne => generated 194 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:51.532002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84aeff-49e1-4020-8859-49f5056c4f11 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.532509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84aeff-49e1-4020-8859-49f5056c4f11 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.532972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc84aeff-49e1-4020-8859-49f5056c4f11 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.534942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 390/772] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:51.536193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84aeff-49e1-4020-8859-49f5056c4f11 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.571235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.582888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0922a179-7ba1-4be3-bfb8-6e6b2d135eb9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.620708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.621397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.621701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.622014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.630580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.632095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e05614b7-858f-44c1-ad67-1e266610ff73 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:51.643818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:51.644730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:51.645639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:51.649808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 383/773] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:51.650966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.651259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.651543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.665372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc84aeff-49e1-4020-8859-49f5056c4f11 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.670290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:51.671519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:51.704956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.705592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.705867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.706243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.706782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.708087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.713721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 391/774] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.722456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.748363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 384/775] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:51.749100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.802905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.840400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:51.867371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d2a5ad-e707-458f-acc7-4286b3d314a6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.876088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dceb9705-be51-42b2-8f18-abbc5abf9790 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:51.918587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.919599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:51.919736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.920148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.920495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:51.920721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] PATH_INFO: `/v3/services/network` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.921121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:51.921503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:51.921805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.922981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:51.923393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:51.924319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.925251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:51.936414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:51.937666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:51.938872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:51.941422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 392/776] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:51.949061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 385/777] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:51.950418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:51.952959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.076300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c5c594-e5d8-4775-875b-1a4f67095113 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.090656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf795f6-4350-49c9-a063-0f5b198ac482 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.136114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.137515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.138085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.138792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.140138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] RBAC: Adding query filter params (name=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.140902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.142510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.146933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.148432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.155160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 393/778] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:52.156988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.157836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.158284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.159720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.160089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=da3e06d942754501b4c903e34e6bbc9d) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.160607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.162884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.163740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.170733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 386/779] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:51 2022] GET /identity/v3/role_assignments?user.id=da3e06d942754501b4c903e34e6bbc9d&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:52.171695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.339322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec281bdb-9b96-4031-bd7a-c686e3693f04 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.388721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.389349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.389622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.389889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.390528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.390826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.392748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.399759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 394/780] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:52.401891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e4484f-49dd-4855-b2f9-0871c4d1ec69 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.410194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.443275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.444127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.444404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.444677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.464180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:52.470826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:52.472188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:52.473207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 387/781] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:52.490749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.502375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.565736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc396bf3-28ff-439b-9ba4-234050305772 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.580485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.615080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.616227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.616815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.617431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.618618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] RBAC: Adding query filter params (interface=public, service_id=f704419f0ad54438ab705ff2dd3f42b2, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.619243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.621280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.634461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 395/782] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/endpoints?service_id=f704419f0ad54438ab705ff2dd3f42b2&interface=public®ion_id=RegionOne => generated 194 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:52.638254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36aaf986-f80b-4a85-a92e-4776de58353d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.639707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.689565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.690142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.690418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.690693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.691397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:52.691688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.693166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.711103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 388/783] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:52.718295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:52.759301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cd631f-715c-450a-8beb-34431dc42c71 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.769644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.810384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.811361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.811837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.812301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.813248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.814846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.826706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 396/784] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/services => generated 772 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:52.882279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b7b6e-a0d7-45ee-b068-2858ea8a884d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:52.902612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:52.927830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:52.928506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:52.928789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:52.929088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:52.947350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:52.948921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:52.974723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:52.975927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:52.976810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:52.976810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:52.976810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:52.976810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 389/785] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:52.986200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:53.119461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecb7581-1a2d-41e1-b20e-b87fb2612994 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:53.163964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:53.164514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.164792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.165087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.165750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:53.166047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:53.167422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d114c05a-2123-4195-827c-352d58fdb039 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:53.185696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 397/786] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:53.193328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:53.258098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed87e4-013b-43a7-9630-b94a40a34de9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.258409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed87e4-013b-43a7-9630-b94a40a34de9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.258669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed87e4-013b-43a7-9630-b94a40a34de9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.260212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 398/787] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:53.266117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039db172-df58-49d7-817e-84541f533cf8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.266488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039db172-df58-49d7-817e-84541f533cf8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.266858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039db172-df58-49d7-817e-84541f533cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.286479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-039db172-df58-49d7-817e-84541f533cf8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:53.287698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-039db172-df58-49d7-817e-84541f533cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:53.329942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:53.365901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15715c66-9183-49d6-b685-6daa387174ca None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:53.405877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:53.407927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.408675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.409168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.417843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:53.419623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:53.419734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 399/788] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:53.439360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:53.440344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:10:53.441292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:53.443900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 390/789] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:10:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:53.450270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039db172-df58-49d7-817e-84541f533cf8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:53.582315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039db172-df58-49d7-817e-84541f533cf8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:53.631774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:53.632775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.633296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.633772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.634724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:53.635233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:53.636874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:53.663808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 400/790] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:10:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:53.670219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:53.713842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:53.793394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:53.826612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:53.827722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a360d54-d2c8-464e-9ee8-0dea9aa4cf7c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:53.866761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:53.867830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:53.868324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:53.868815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:53.943017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:53.945047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:53.982264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.053182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 391/791] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:10:53 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:10:54.055167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1794a75-87c9-4bf2-820d-699cb02a58de None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.058452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.115385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.116118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.116298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.116566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.155962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:54.161571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:54.162505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:10:54.163407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:54.169391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 401/792] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 376 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:54.170309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.172251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a946c7ec-63ab-4db1-b1fd-c796f0f12e81 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.214222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.221188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.222244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.222760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.223200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.234587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:54.236339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:54.253077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:54.254126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:54.255016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:54.255016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:54.255016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:54.257227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 392/793] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:54.262445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.321049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.339253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c238789-d8fb-4929-bb9c-c07bf4c966ed None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.376994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.377731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.378015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.378392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.379076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:54.379379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:54.380756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a60fd8-58ec-43ff-b2f3-eed1ba812cbf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:54.390592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 402/794] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:54.392621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56769a-12aa-4a87-a8c2-82c0cc68db6c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.392911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56769a-12aa-4a87-a8c2-82c0cc68db6c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.393212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56769a-12aa-4a87-a8c2-82c0cc68db6c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.393857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff1bb54-574b-412c-919a-6c31972bdd25 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.394734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 403/795] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:54.400884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56769a-12aa-4a87-a8c2-82c0cc68db6c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.440209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.440830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.441141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.441676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.442329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:54.442871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:54.444911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c19cb80-ffcf-4cf6-94e2-b22a335bb3e1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:54.460828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 393/796] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:54.462780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.463119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.463394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.489402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:54.490541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:54.544764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.552249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56769a-12aa-4a87-a8c2-82c0cc68db6c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.596327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 394/797] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:54.596327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.616599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.618041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.618780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.619249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.649151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:54.650670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:54.652779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:54.678828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:54.679829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:54.680743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:54.680743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:54.680743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:54.680743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 404/798] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:54.686698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.786301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd29d628-37e6-428f-80a0-2fd39e509ab7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.829559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.830539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.830820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.831356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.838223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d484a0a3-f59f-457c-857b-d508ff37ff6c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:54.841415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:54.842882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-94d9f84a-c621-499b-8dd7-121f1178fc2e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:54.859282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:54.860267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:54.861198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:54.861625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 395/799] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:54.863766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8299c4-ad9f-4ee1-aeaf-1e330f5f1fed None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.864481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8299c4-ad9f-4ee1-aeaf-1e330f5f1fed None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.864761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8299c4-ad9f-4ee1-aeaf-1e330f5f1fed None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.866818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 396/800] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:54.868369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8299c4-ad9f-4ee1-aeaf-1e330f5f1fed None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:54.878275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:54.878984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.879334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.879678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.880401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:54.880778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:54.882254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2367f02-1b67-48e0-82c7-f2c0d7814e5c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:54.895662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 405/801] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:54.901746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:54.902115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:54.902472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:54.910218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.921533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:54.922950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:54.976137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:54.988516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8299c4-ad9f-4ee1-aeaf-1e330f5f1fed None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.005038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 406/802] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:55.005038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.037218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.037849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.038130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.038404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.039212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.039524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.041141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.055557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 397/803] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.056524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.102661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d9bfa6-4c9a-47c3-be3e-38e41a708b23 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.140103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.141534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.142252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.142734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.152502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.154295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.173876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:55.176417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:55.177380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:10:55.178305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:55.180971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 407/804] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:10:54 2022] GET /identity/v3/users/swift => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:55.182251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.184395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96516813-6366-428c-b699-fa7e45a1968e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.235195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.236140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.236789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.237099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.237369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.237956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.239961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:55.249525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:55.250762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:55.251829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 398/805] 10.222.0.73 () {56 vars in 1129 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:55.253588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.303912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f7c654-d66a-4bd1-8e10-6fee3626307e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.336800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.337428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.337703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.337976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.338690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=059f72c1793e492fb052bebbf29b5bec) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.338993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.340410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c854676c-6c4e-4413-a46d-0beb676eb420 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.346679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 408/806] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/role_assignments?user.id=059f72c1793e492fb052bebbf29b5bec&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.347914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0cb5b-307f-40f7-be49-44cf99570cbc None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.348197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0cb5b-307f-40f7-be49-44cf99570cbc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.348466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0cb5b-307f-40f7-be49-44cf99570cbc None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.354174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 409/807] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:55 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:55.355018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0cb5b-307f-40f7-be49-44cf99570cbc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.367894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be179151-b20a-4fd1-957e-4b423a6a9557 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.410257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.410882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.411164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.411429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.412172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.412474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.413965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e5bccd-30e7-4093-9609-29b5d9f7c203 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.484206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0cb5b-307f-40f7-be49-44cf99570cbc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.496201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.526097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.526667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.526944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.527214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.527856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.528159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.529579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.536913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 410/808] 10.222.0.73 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.537875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.558375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 399/809] 10.222.0.73 () {56 vars in 1135 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.558375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.558375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.558375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.576809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:55.578161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:55.589211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.661056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 400/810] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:55.661056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.677223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d57eecb0-9f19-4f4f-b362-8d5b98a1987c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.717086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.718101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.718567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.719601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.731358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:10:55.737685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:55.738641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:10:55.739478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:55.741983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 411/811] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:55.743139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.782015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.806229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1cb31c-042c-4672-b2ea-1086d5408a87 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.851417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.852655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.853304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.853776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.854740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:55.855398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.857311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.865145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 401/812] 10.222.0.73 () {56 vars in 1124 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:55.866456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.872415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b88106-0eeb-4f97-8dfd-47e0a3f799ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:55.881410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:55.921050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:55.922016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:55.923371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:55.923649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:55.942149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:55.943968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:55.964341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:55.965652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:55.966792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:55.966792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:55.966792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:55.966792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 412/813] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:55.966792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:55.993486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9299e-8766-479d-a33b-b90abd740327 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.032601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.033553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.033832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.034110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.034783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.035081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.036453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.045656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 402/814] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:10:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:56.046515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.073283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.091618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628645f7-dc67-49fe-acc3-77dc8976a922 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.129052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.132342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.132955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.133266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.133534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.134372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.135553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.162555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce56db75-f7ff-4e21-9ea4-c19ee42e37c5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.198196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.198822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.199102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.199374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] PATH_INFO: `/v3/services/network` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.199960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.201394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:56.210696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:56.212133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:56.213271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 403/815] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:56.214311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.233384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 413/816] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:56.233384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.377108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8267710d-feca-4ca7-829a-fcdaa047951f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.409598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1418ee1b-900c-4a47-adb0-2bf61f168660 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.425750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.427898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.428593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.429077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.430040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.430486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.432357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.434262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.450454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 404/817] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:56.456188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.458617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.459290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.459572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.460050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.465203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.478479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.480108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:56.499921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:56.501492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:56.502932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:56.502932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:56.502932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:56.502932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 414/818] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:56.502932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.586412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf9fa1c-9ee4-4e58-8d76-1198c325d4f2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.625680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.626535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.626832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.627108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.627846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] RBAC: Adding query filter params (name=network) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.628158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.629578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.634889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 405/819] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services?name=network => generated 124 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:56.635467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.650015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4a321d-c5a4-4b63-9d75-9a259e88463c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.688829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.689486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.689777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.690053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.697666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.699124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.710745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:56.711855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:56.712928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 415/820] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:56.721819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.771887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b196f0d-21d8-4962-9ec1-96241bb0de5b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.777456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:56.812303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.813124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.813353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.813629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.814433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.814682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.816314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.830219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 406/821] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:56.835653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.841262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a997f1-fb32-4cff-afff-4320ccd0169a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.880033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:56.881116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:56.881606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:56.882083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:56.882925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] RBAC: Adding query filter params (type=network) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:56.883454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:56.885276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:56.892545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 416/822] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:56.893939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:56.960386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0b5e98-f772-4963-ad7d-491b6b8cc734 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:56.982625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:57.004488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:57.005143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.005418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.005688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.006771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:57.007257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:57.008835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:57.009706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b06659c-446a-42fa-a822-b04d5d120add None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:57.023520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 407/823] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:57.024650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:57.036077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:57.067062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:57.068494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.068842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.069158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.079052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:57.081568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:10:57.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:57.098248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:10:57.099202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:57.103882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 417/824] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:10:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:57.104916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:57.142824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19d56e3-4afe-43bf-b44c-a5129e9b3069 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:57.181703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:57.182691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.183253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.183844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.184714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:57.186426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:57.230266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27826b8-abc8-4e2d-956b-ac8d289893f9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:57.268968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a47e9b-36ce-4d4a-8ca0-2e6eb7559dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:57.269698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47e9b-36ce-4d4a-8ca0-2e6eb7559dcf None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.269956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47e9b-36ce-4d4a-8ca0-2e6eb7559dcf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.270231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a47e9b-36ce-4d4a-8ca0-2e6eb7559dcf None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/a54377901631412abe887223a71d3623/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.272207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47e9b-36ce-4d4a-8ca0-2e6eb7559dcf None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=a54377901631412abe887223a71d3623, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:57.273716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a47e9b-36ce-4d4a-8ca0-2e6eb7559dcf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:57.329048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 408/825] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:10:57 2022] POST /identity/v3/endpoints => generated 337 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:10:57.329048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:57.372583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:57.382276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 418/826] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:57 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/a54377901631412abe887223a71d3623/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:57.383723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ec0cb-5644-48ad-9ec7-23be8ce64829 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.384072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ec0cb-5644-48ad-9ec7-23be8ce64829 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.384460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ec0cb-5644-48ad-9ec7-23be8ce64829 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.386206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 419/827] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:57.392493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.392881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.393107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.418133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:57.418133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:57.456348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:57.548057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3143345-bec3-40a5-a759-bdc4ed5a179c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:57.559435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 420/828] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:57 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:57.601979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:57.602911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.603189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.603457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.604199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:57.604555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:57.606037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:57.626329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 409/829] 10.222.0.73 () {56 vars in 1149 bytes} [Tue Aug 23 22:10:57 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:57.632065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:57.741919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591dcd22-0208-42e3-8fdf-8011feaa0456 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:57.778235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:57.778932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:57.779214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:57.779492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:57.791127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:57.792587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:10:57.807925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:57.808892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:10:57.809812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:10:57.809812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:10:57.809812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:57.809812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 421/830] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:57.818133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:57.839875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:57.856530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:57.945030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:57.958466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fe50dc-97ab-4d36-9aa8-46351145e56d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:57.962550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cd0624-3156-4f0b-8620-9fe116f09b24 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.006180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.006547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.007259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.007547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.007831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.008016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.008433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.008690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.009207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.010209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.010558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:58.011036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.012583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:58.022220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:58.023315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:58.029484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 422/831] 10.222.0.73 () {56 vars in 1132 bytes} [Tue Aug 23 22:10:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:10:58.030624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 410/832] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:58.032168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.037584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.137556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.143446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddecd8d3-8280-4855-8cfd-9e4e97d7fe02 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.179600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c6eda9-aff2-4703-b270-ec80bc5e1bd7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.183420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.184055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.184329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.184597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.193030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.194492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:10:58.207057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:58.208183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:10:58.209367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:10:58.213325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 411/833] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:58.214231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.225192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.225982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.226370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.227289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.228344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.229642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:58.230371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.232102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.239528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 423/834] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:58.244628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.338417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da54c5b-8f06-4ccb-b66d-496fcc70edf6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.377217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01773d08-4749-442a-a3c3-f7d31c30d33e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.381750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.382902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.383388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.383878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.385871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:58.386413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.389790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a8fa3a-0c44-4434-8525-82f814f19e93 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.402935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 412/835] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:58.404017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d828b-8189-48e9-adab-387a98f30e11 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.404302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d828b-8189-48e9-adab-387a98f30e11 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.404574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237d828b-8189-48e9-adab-387a98f30e11 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.405987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 413/836] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:58 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:58.410252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d828b-8189-48e9-adab-387a98f30e11 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.419602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.420325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.420699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.421117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.421841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:58.422286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.423746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2166f1-0c60-45b7-9886-72130143885d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.428436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 424/837] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:58.429812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df196e81-92eb-450c-9793-376727ecab49 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.430099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df196e81-92eb-450c-9793-376727ecab49 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.430375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df196e81-92eb-450c-9793-376727ecab49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.437001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.450795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-df196e81-92eb-450c-9793-376727ecab49 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:58.452185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-df196e81-92eb-450c-9793-376727ecab49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:58.530502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.541454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 425/838] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:58 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:10:58.576302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237d828b-8189-48e9-adab-387a98f30e11 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.618989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.619660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.619942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.620218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/da3e06d942754501b4c903e34e6bbc9d/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.637873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=da3e06d942754501b4c903e34e6bbc9d, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.639366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.730867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 414/839] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:10:58 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/da3e06d942754501b4c903e34e6bbc9d/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:10:58.817088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df196e81-92eb-450c-9793-376727ecab49 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:58.928665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df196e81-92eb-450c-9793-376727ecab49 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:58.961940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:58.967458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:58.968109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:58.968384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:58.968651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:58.970014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:58.971420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:10:58.981224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:10:58.982224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:10:58.983071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 426/840] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:10:58.989486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:59.111286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fac5111-40e0-442f-9c93-57c306ab3cb7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:59.131657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:59.150055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:59.150677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.150956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.151225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.151866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:59.152171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:59.153576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a21fe27-d84a-42ee-8f77-a55b619fea0d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:59.159350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 415/841] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:10:59.164921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:10:59.278937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baffa5e2-1726-44ab-907e-5d408201170b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:10:59.316178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:10:59.317194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.317700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.318173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.319049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:10:59.319546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:10:59.321140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47273cb0-a4e6-40d9-8544-3fdba7d702fc None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:10:59.326593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 427/842] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:10:59 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:10:59.638176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b499c-7d54-42e5-87a3-497657311894 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.638543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b499c-7d54-42e5-87a3-497657311894 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.638749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565b499c-7d54-42e5-87a3-497657311894 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.640573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 416/843] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:10:59.646913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:10:59.647203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:10:59.647481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:10:59.666300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:10:59.667480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:10:59.731184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:10:59.742917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 428/844] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:10:59 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:00.017036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b499c-7d54-42e5-87a3-497657311894 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.120194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565b499c-7d54-42e5-87a3-497657311894 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.156687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.157828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.158369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.158638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.169184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:00.173819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:00.174755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:00.175585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:00.179835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 417/845] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:00.182085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.287770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a64948-8521-419c-a1c0-3d320119a2c0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.322532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.323143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.323421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.323713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.324401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:00.324701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:00.326122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5229cb88-7345-42be-a576-3981bc5689d4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:00.335277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 429/846] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:00.340613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.390407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:00.452612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162ae462-561b-4d07-9f83-1069d778d3f6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.487119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e279-f3f9-416f-b714-1970a1030900 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.487468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e279-f3f9-416f-b714-1970a1030900 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.487671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f6e279-f3f9-416f-b714-1970a1030900 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.493105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 430/847] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:00.494302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.494583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.494852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.502573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.503496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.503976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.504461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.513402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:00.514584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:00.517361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:00.519176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:00.539704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:00.540678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:00.541608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:00.541608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:00.541608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:00.541608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 418/848] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:00.556708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.599182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:00.639396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 431/849] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:00.700343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a250d-1601-4088-adea-53e3616f0fcd None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.713461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:00.738796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.739713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.740180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.740601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.741491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:00.741944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:00.743835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd219efd-498a-4dfb-9e69-edc42681c052 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:00.757786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 419/850] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:00.762995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.865362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a312-417f-4ad8-b884-af5a0a4b5fe3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.865550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a312-417f-4ad8-b884-af5a0a4b5fe3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.865910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a8a312-417f-4ad8-b884-af5a0a4b5fe3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.868381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 420/851] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:00.872499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.872781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.873089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.873398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a610323-ef7d-48ba-958e-e87af361f1a3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:00.892822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:00.894057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:00.920175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:00.921197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:00.921625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:00.921936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:00.930131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:00.931587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:00.944758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:00.945807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 432/852] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:00.950078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:00.971056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 421/853] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:00 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:00.972497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.004459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.082737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a60332-5eb5-45da-8e57-6a750decd5c3 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.096395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.119658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608a01b9-bce1-43a8-9fa1-98b35ecb37d0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.121885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.122766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.123253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.123748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.124600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:01.127285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:01.159744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.160411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.160694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.161005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.161761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:01.162067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:01.163485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:01.187214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 422/854] 10.222.0.73 () {56 vars in 1135 bytes} [Tue Aug 23 22:11:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:01.187835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 433/855] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:00 2022] POST /identity/v3/services => generated 269 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:01.188369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.239656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.338379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd76671c-3e7f-4c1a-8243-f4f759094e36 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.367437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.374701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee22eb0-3fb6-475a-b32b-077ca8d45051 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.387131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6964742-87b4-4de0-9213-3e80f4d607e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.388354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6964742-87b4-4de0-9213-3e80f4d607e6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.388738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6964742-87b4-4de0-9213-3e80f4d607e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.389103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6964742-87b4-4de0-9213-3e80f4d607e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.405075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.424717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2ed1b-aee9-47af-b8a8-bb530dcb2257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.425504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2ed1b-aee9-47af-b8a8-bb530dcb2257 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.425786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2ed1b-aee9-47af-b8a8-bb530dcb2257 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.426138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2ed1b-aee9-47af-b8a8-bb530dcb2257 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.441562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2ed1b-aee9-47af-b8a8-bb530dcb2257 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-59d2ed1b-aee9-47af-b8a8-bb530dcb2257 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:01.448096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:01.449144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:11:01.449999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:01.453829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 434/856] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:01.455665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2735f-fb29-47a1-b770-c1c8a64a8f5c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.455952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2735f-fb29-47a1-b770-c1c8a64a8f5c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.456226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc2735f-fb29-47a1-b770-c1c8a64a8f5c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.457970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 435/857] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:01.458445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2735f-fb29-47a1-b770-c1c8a64a8f5c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.480019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6964742-87b4-4de0-9213-3e80f4d607e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:01.481538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6964742-87b4-4de0-9213-3e80f4d607e6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:01.583405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc2735f-fb29-47a1-b770-c1c8a64a8f5c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.613387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 423/858] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/auth/tokens => generated 1271 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:01.615742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.616032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.616307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.625553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.626090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.626370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.626634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.627290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:01.627593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:01.628958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:01.644027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 436/859] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:01.644214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:01.644320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.644405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:01.668974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.683073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.722656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 424/860] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:01 2022] POST /identity/v3/auth/tokens => generated 1271 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:01.723535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.792315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ec1ac2-8577-49d5-9876-f21be5fee0ff None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.835111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.835765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.836121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.836404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.848991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:01.850655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:01.853350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbf7dfe-5e40-41c9-aedf-494886ed9e2d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:01.867251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:01.868285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:01.869205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:01.869205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:01.869205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:01.869205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 437/861] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:01.880493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.905566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:01.906680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:01.907184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:01.907661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:01.922393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:01.924344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:01.944316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:01.945451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:01.946332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:01.946332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:01.946332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:01.953972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 425/862] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:01.955955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:01.989504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:01.991778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.005617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675c585f-fc79-4015-8edf-d73d6a3b8177 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.044880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.045515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.045832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.046104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.047270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.047566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.048946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.060940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 438/863] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:02.061948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.079122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3667e18c-abca-410c-8c43-4bc665953924 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.115762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.117209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.117745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.118185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.119070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.119545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.121267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.135883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 426/864] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:02.136639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.174837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943ddbc0-afba-47a7-b951-22498d5f0548 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.205581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.210509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.211124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.211406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.211681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.212337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.213747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.227337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.243411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588f160-f2ed-4e45-9a7b-688423fedb1a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.278845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.279483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.279763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.280035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.287214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.288647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:02.300924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:02.307905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 427/865] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:02.308925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.309939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 439/866] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:02 2022] POST /identity/v3/services => generated 251 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:02.314247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.445191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a398da0-265e-480f-b6c9-f8661641ab10 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.463807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1164779-e427-4fde-b8e8-77e77970ec29 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.498623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.500757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.501473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.501748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.506686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.506827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.507041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.507317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.508022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.508329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.509999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.511001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.513684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.524844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 440/867] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:02.527532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:02.528481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:02.529498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 428/868] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:02.531452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.538028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.549174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.609455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.649945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d7219-b8bb-47cf-99a6-0244cd1abc1d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.671744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae8b89f-e5bb-4e5e-a92e-c02e02c8e7e6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.690612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.691500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.691962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.692409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.693385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.693885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.695632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.720125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 429/869] 10.222.0.73 () {56 vars in 1149 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:02.722492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.723172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.723532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.723940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.724226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:02.724960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=a54377901631412abe887223a71d3623) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:02.725415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.727328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:02.737251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 441/870] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/role_assignments?user.id=a54377901631412abe887223a71d3623&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:02.804564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:02.851079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8be7e69-3073-44dc-aeac-b649738c31d8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:02.891255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cd4c27-3f07-448d-8ac9-3bebd1738784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:02.891255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4c27-3f07-448d-8ac9-3bebd1738784 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:02.891255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4c27-3f07-448d-8ac9-3bebd1738784 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:02.891615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cd4c27-3f07-448d-8ac9-3bebd1738784 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:02.967062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd4c27-3f07-448d-8ac9-3bebd1738784 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:02.969727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cd4c27-3f07-448d-8ac9-3bebd1738784 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.032362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.073072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 430/871] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:02 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:03.078734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.189537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d329ef71-a509-4929-9ed7-5d588ba08460 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.204953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.227622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.229063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.229552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.230027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.241655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.243580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:03.259648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:03.260645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:03.261580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:03.261580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:03.261580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:03.263713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 442/872] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:03.264827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630f0a-48fe-48cb-81ad-850a6eca92f1 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.265400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630f0a-48fe-48cb-81ad-850a6eca92f1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.265570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22630f0a-48fe-48cb-81ad-850a6eca92f1 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.269335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 431/873] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:03.270494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.273082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.273372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.273652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.293689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:03.294891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:03.374172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 432/874] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:03 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:03.388206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257c1282-1677-41b1-8c1f-3d06a3642b77 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.426289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.427307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.427811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.428300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.429174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:03.429709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.431335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.445406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 443/875] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:03.450557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.553904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.561191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c62fae0-6b37-4f89-90bf-6877b1a9a2b7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.597956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.598600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.598884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.599161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.606522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.607965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:03.620833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:03.621772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:03.622805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 433/876] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:03.630274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.660644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.676992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.752606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ff6956-77cc-4e36-ba83-754e06a56ec9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.798341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea9a840-e5ec-422f-be46-697da85e783f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.809341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.810740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.811437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.812105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.814244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:03.815321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.818821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.837356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:03.838157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:03.838507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:03.838888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:03.839546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:03.840992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:03.845998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 444/877] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:03.854444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:03.854677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:03.856396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:03.857540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 434/878] 10.222.0.73 () {56 vars in 1132 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:03.865593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:03.925588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.972048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:03.986204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b457ecb4-1969-43ac-b2de-82f5966dfaf6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:03.991649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd4b6ca-5e16-435f-b46f-4a399e289097 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.025254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.026114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.026409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.026681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.027318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:04.027612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.028962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.030759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.032236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.032748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.033477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.034372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=da3e06d942754501b4c903e34e6bbc9d) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:04.034855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.035350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 435/879] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:04.036655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.040670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.053045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 445/880] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:03 2022] GET /identity/v3/role_assignments?user.id=da3e06d942754501b4c903e34e6bbc9d&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:04.139070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:04.161411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55a9666-ca44-450e-bbb7-f2893af3aa8b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.200607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.201256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.201539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.201813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.202516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:04.203565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.204948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.210811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 436/881] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:04.216810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.337664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e64df6-7c22-4f4f-8549-3e6dbb8729ec None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.380134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.380755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.381057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.381341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.382055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] RBAC: Adding query filter params (interface=public, service_id=6c98062112404537b30ff456d186f58f, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:04.382367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.383815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4dea8c-6437-464f-83c2-3a9d9706efe5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.390113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 446/882] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/endpoints?service_id=6c98062112404537b30ff456d186f58f&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:04.396675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.518284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca829-878c-4087-8f0e-e2bced3ecf6d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.518650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca829-878c-4087-8f0e-e2bced3ecf6d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.518822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3ca829-878c-4087-8f0e-e2bced3ecf6d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.520409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 447/883] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:04.525690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9a84-d6b7-46cb-9dcd-706a3b85fdf4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.525974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9a84-d6b7-46cb-9dcd-706a3b85fdf4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.526249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14d9a84-d6b7-46cb-9dcd-706a3b85fdf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.529828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a99929-d5ef-4fea-b8c5-13a8bad98550 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:04.546097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c14d9a84-d6b7-46cb-9dcd-706a3b85fdf4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:04.547402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c14d9a84-d6b7-46cb-9dcd-706a3b85fdf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:04.563216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:04.576412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da0f8c2-dd4c-426b-b0a8-bdb092839236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:04.577597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0f8c2-dd4c-426b-b0a8-bdb092839236 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.578595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0f8c2-dd4c-426b-b0a8-bdb092839236 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.579027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da0f8c2-dd4c-426b-b0a8-bdb092839236 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.579969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0f8c2-dd4c-426b-b0a8-bdb092839236 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:04.582359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da0f8c2-dd4c-426b-b0a8-bdb092839236 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:04.591509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 437/884] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/services => generated 1268 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:04.661807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 448/885] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:04.858037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d66436-a638-4b87-9fcc-cc9d2490c0a3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.858451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d66436-a638-4b87-9fcc-cc9d2490c0a3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.858648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d66436-a638-4b87-9fcc-cc9d2490c0a3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.860294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 438/886] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:04.866047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:04.866339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:04.866609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:04.885767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:04.886984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:04.943343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d66436-a638-4b87-9fcc-cc9d2490c0a3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:04.956047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:04.959493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 449/887] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:04 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:05.057621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d66436-a638-4b87-9fcc-cc9d2490c0a3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.093608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.094571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.094995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.095409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.096152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.097807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:05.107482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:05.108706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:11:05.110560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:05.112932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 439/888] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:11:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:05.118841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.194440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.226645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5312a144-7ea1-47c2-877e-d51bf34d0baf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.240423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.264838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.265549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.265836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.266113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.266746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.267047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.268644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.274939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 450/889] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:05.281368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.326562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.358419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba558a8-4bb8-4340-a388-35ae60fc058e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.394546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b14169b-ef48-4642-ac8b-07b269a0d721 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.397193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.398376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.398856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.399345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.400122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.401716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:05.412355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:05.413692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:05.414536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:05.417452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 440/890] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:05.423247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.433498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.434110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.434441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.434716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.435348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.435647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.437684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.440272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.448566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 451/891] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:05.449864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.551828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4efcdc-7a4f-4f48-a295-5f0961335f6d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.572445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659ccf9e-baa8-4c1b-b0b9-54af7767aa2d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.597882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.598631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.599278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.599278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.600068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.600463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.601951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.604380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.608239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 441/892] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:05.614160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.618972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.619609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.619898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.620171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.620885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] RBAC: Adding query filter params (interface=public, service_id=b161781ee356418e922db93a6c34e0db, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.621238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.622611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.634286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 452/893] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/endpoints?service_id=b161781ee356418e922db93a6c34e0db&interface=public®ion_id=RegionOne => generated 194 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:05.635688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:05.711788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:05.755125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af852196-ff25-4d47-b20a-4beb9ac87c3d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.771789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e3eff0-9825-410f-9698-1f7697816ff0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:05.803276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.804506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.805265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.806175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.807283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:05.807797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.810517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98fa35-7db8-49b3-b0d7-148d5f86d8cf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.814252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0275f4c2-50b2-408d-b5db-dc6d171f04d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:05.814872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275f4c2-50b2-408d-b5db-dc6d171f04d3 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:05.815190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275f4c2-50b2-408d-b5db-dc6d171f04d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:05.815423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0275f4c2-50b2-408d-b5db-dc6d171f04d3 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:05.816037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275f4c2-50b2-408d-b5db-dc6d171f04d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:05.817455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0275f4c2-50b2-408d-b5db-dc6d171f04d3 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:05.817891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 442/894] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:05.829186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 453/895] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:05 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:06.321868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b6b0-687f-442c-95be-c9d02c82642f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.322517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b6b0-687f-442c-95be-c9d02c82642f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.322990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b1b6b0-687f-442c-95be-c9d02c82642f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.324770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 443/896] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:06.330932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2822a54-eb79-403e-ae2e-624f5aa0540b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.331529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2822a54-eb79-403e-ae2e-624f5aa0540b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.332016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2822a54-eb79-403e-ae2e-624f5aa0540b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.352068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d2822a54-eb79-403e-ae2e-624f5aa0540b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:06.353664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d2822a54-eb79-403e-ae2e-624f5aa0540b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:06.440182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 454/897] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:06.612267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e03d-2878-4465-b0ad-0a67398d2b34 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.613256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e03d-2878-4465-b0ad-0a67398d2b34 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.613755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a0e03d-2878-4465-b0ad-0a67398d2b34 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.615525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 444/898] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:06.620996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.621307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.621583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.641825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:06.643427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:06.704403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.720973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 455/899] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:06 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:06.732793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e03d-2878-4465-b0ad-0a67398d2b34 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:06.805558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:06.853750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a0e03d-2878-4465-b0ad-0a67398d2b34 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:06.889383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:06.889951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:06.890758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:06.891034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:06.902339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:06.908709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:06.909695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:06.910800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 445/900] 10.222.0.73 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:06.920439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.000237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.038392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e24402-b1e2-4e4b-ae6c-db76f57ac482 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.078865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.079480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.079765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.080038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.080819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.081173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.082663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.092309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 456/901] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:07.098813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.128572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979ce672-8954-4c80-9c69-65a457e597f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.159960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.161460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.173666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.174547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.175406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.175406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.175733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.177673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:07.190202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:07.191421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 23 22:11:07.192516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:07.203947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 446/902] 10.222.0.73 () {56 vars in 1132 bytes} [Tue Aug 23 22:11:06 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:07.210976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.232132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631a324c-1719-4dda-b681-9f29af8e9aa6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.278878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.279870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.280430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.281574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.299581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.301549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:07.317743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:07.318975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:07.320005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:07.320005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:07.320005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:07.326704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 457/903] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:07.329005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.346524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d15fa9-2514-4378-8f01-dc0888b9281f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.399072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.400636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.401263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.401687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.402481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.402934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.404488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.411372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 447/904] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:07.416677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.472112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4c07c7-bdc4-4db3-b835-abc51e9ca85a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.474245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.504453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.529987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.530614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.531574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.532090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.533174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.533486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.534910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.561096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 458/905] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:07.562807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.584993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad99c97-9a07-4137-b6a4-28622ac95d7c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.627409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.628857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.629374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.629882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.630725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:07.631266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.632856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.638953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 448/906] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:07.644558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.707324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b720ee00-f0c1-46e6-b2c7-dbc22f492e1c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.760684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.761725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.762174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.762613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.774379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.776301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:07.791263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:07.792301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:07.793486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:07.801449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdf9b4-2d0b-4f09-a81a-f4bf6ab21c70 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.801949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 459/907] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:07.803411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:07.821133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.833407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:07.852685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e937752-5fe3-4df7-acb2-e07ace329876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:07.853456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e937752-5fe3-4df7-acb2-e07ace329876 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.853947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e937752-5fe3-4df7-acb2-e07ace329876 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.854295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e937752-5fe3-4df7-acb2-e07ace329876 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.855620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e937752-5fe3-4df7-acb2-e07ace329876 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:07.857114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e937752-5fe3-4df7-acb2-e07ace329876 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:07.919952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 449/908] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:07 2022] POST /identity/v3/endpoints => generated 347 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:07.920905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcab0bb-5a2d-48e0-82d8-27a361e68b03 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.921313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcab0bb-5a2d-48e0-82d8-27a361e68b03 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.921608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adcab0bb-5a2d-48e0-82d8-27a361e68b03 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.928964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 450/909] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:07.928964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aba443-9939-437a-b2f5-e464f82dd5e4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.928964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aba443-9939-437a-b2f5-e464f82dd5e4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.928964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aba443-9939-437a-b2f5-e464f82dd5e4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.928964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 451/910] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:07.933794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:07.934822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:07.935333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:07.942507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5933afc6-de10-4c84-a02e-d4e029301b8a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:07.962340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:07.963457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:08.007291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:08.007920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.008201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.008520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.009406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:08.009684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:08.012509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe213e-8103-4ee5-8744-97ad08b9a696 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:08.037697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 460/911] 10.222.0.73 () {56 vars in 1149 bytes} [Tue Aug 23 22:11:07 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:08.039079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.039367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.039638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.063602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 452/912] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:07 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:08.064206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:08.065271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:08.065518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:08.161764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 461/913] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:08 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:08.206626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f977a733-576f-46dd-9a20-5e0cf994b11c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:08.208748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:08.246339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:08.247142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.247491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.247913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.323246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:08.325173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:08.397700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:08.411274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:08.442622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 453/914] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:08 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:08.448108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:08.451380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:08.532484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cef62e-1800-4260-b4ed-e7358deda6ca None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:08.572949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:08.573610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.573885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.574155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.574819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:08.576192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:08.594503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd185d8-ec15-49ed-9547-12adc79b41f9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:08.628761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:08.629596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.629872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.630160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.643342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:08.644776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:08.653669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 462/915] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:08 2022] POST /identity/v3/services => generated 241 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:08.654798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:08.662491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:08.663561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:08.664436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:08.664436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:08.664436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:08.664436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 454/916] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:08.672343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:08.784782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:08.794397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:08.841897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3bf371-1cc4-4a31-a62c-3212c10f29be None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:08.853385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9018f-5ad3-4e98-a6ef-df462d7dea7e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:08.895519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:08.896173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.896488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.896772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.897551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:08.900088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:08.912836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:08.913488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:08.913772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:08.914044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:08.914732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:08.915033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:08.915463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:08.916374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 23 22:11:08.917588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:08.918321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:08.918391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 463/917] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:11:08 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:08.926285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:08.932640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 455/918] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:08.938484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:09.044490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0d5119-24c3-4914-b531-d7f98d691bde None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:09.054768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febe5c13-687c-4c75-9677-c21ef97869aa None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:09.072189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:09.075899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:09.085448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:09.086321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.086744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.087157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.087937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:09.088378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:09.089915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:09.095587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 464/919] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:11:08 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:09.095868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:09.096514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.096794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.103863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.104073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:09.105450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:09.107019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:09.121754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:09.122890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:09.123957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 456/920] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:09.129677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:09.210280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acede65a-5a82-4bfa-b173-122958a36527 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:09.247470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a20de32-9de7-4952-bddc-0c4331cda62d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:09.247901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:09.248521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.248803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.249111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.249785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:09.250289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:09.251735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:09.261925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 465/921] 10.222.0.73 () {56 vars in 1114 bytes} [Tue Aug 23 22:11:09 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:09.263297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:09.283885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:09.284504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.284787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.285105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.285810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:09.286115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:09.287537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:09.307293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 457/922] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:09.308143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:09.325104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:09.362873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:09.375275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b3f24b-3c98-468b-abc0-d7bab849d582 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:09.425790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77099938-9349-4ac2-9eb1-84616556b2cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:09.427129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099938-9349-4ac2-9eb1-84616556b2cd None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.427681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099938-9349-4ac2-9eb1-84616556b2cd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.428105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77099938-9349-4ac2-9eb1-84616556b2cd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.429107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77099938-9349-4ac2-9eb1-84616556b2cd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:09.431288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77099938-9349-4ac2-9eb1-84616556b2cd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:09.435692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e37dea8-2265-4637-b370-db5b60b48509 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:09.483341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:09.484428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:09.484937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:09.485298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:09.486152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] RBAC: Adding query filter params (role.id=54a2522e3f374c3e8b4ff879dc696487, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=da3e06d942754501b4c903e34e6bbc9d) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:09.486466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:09.488013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f59cedf-a747-48ec-af39-b99bad2dc868 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:09.497868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 458/923] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:09 2022] GET /identity/v3/role_assignments?user.id=da3e06d942754501b4c903e34e6bbc9d&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=54a2522e3f374c3e8b4ff879dc696487 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:09.525295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 466/924] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:09 2022] POST /identity/v3/endpoints => generated 346 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:10.076658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cad46-6cfb-4656-a48b-4a450040351e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.078260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cad46-6cfb-4656-a48b-4a450040351e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.078764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cad46-6cfb-4656-a48b-4a450040351e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.080602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 459/925] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:10.086091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847dfd1-0d94-4691-829a-697db208771f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.086927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847dfd1-0d94-4691-829a-697db208771f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.087392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c847dfd1-0d94-4691-829a-697db208771f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.105744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c847dfd1-0d94-4691-829a-697db208771f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:10.107211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c847dfd1-0d94-4691-829a-697db208771f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:10.181756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 467/926] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:10.459210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cad46-6cfb-4656-a48b-4a450040351e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.569565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.575157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cad46-6cfb-4656-a48b-4a450040351e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.613367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.614376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.614883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.615352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.616144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.617750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ab7973a6-dc19-40c9-b8c6-edc217189a86 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:10.627364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:10.628396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:10.629253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:10.631827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 460/927] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:10.637773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847dfd1-0d94-4691-829a-697db208771f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.680445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.716413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d696d7-31c9-4473-a98d-84f6b1ffabe6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.716697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d696d7-31c9-4473-a98d-84f6b1ffabe6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.717079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d696d7-31c9-4473-a98d-84f6b1ffabe6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.719328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 461/928] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:10.724737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.725136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.726088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.747212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:10.748526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:10.766148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c847dfd1-0d94-4691-829a-697db208771f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.807875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.808668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.808915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.809281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.810051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:10.810415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.811880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.817402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 468/929] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:10.822578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:10.831195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 462/930] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:10 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:10.940617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:10.944276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db9e7c0-ecbd-4dfd-8f05-fdfa4f0acf63 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:10.981044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:10.982055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:10.982534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:10.982997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:10.983924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:10.984925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:10.986531 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:10.991809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 469/931] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:10 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:11.114313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.217074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.227146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43249822-18b8-4936-be6c-2d24c86bffde None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.279941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70731321-31c2-4351-94b8-834a1307778b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.281763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70731321-31c2-4351-94b8-834a1307778b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.282416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70731321-31c2-4351-94b8-834a1307778b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.283364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70731321-31c2-4351-94b8-834a1307778b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.284145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70731321-31c2-4351-94b8-834a1307778b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.285780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70731321-31c2-4351-94b8-834a1307778b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-70731321-31c2-4351-94b8-834a1307778b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:11.298097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:11.299829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:11.301273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:11.304812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 463/932] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:11.310671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.423178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd580c34-6079-416f-bfa0-f0803cf8e8f2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.460902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.461562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.461844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.462115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.462754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:11.463056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.464428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b3753e-715f-4de4-8bda-c726d5d87180 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.475172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 470/933] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:11.476519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70731321-31c2-4351-94b8-834a1307778b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.518813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d918-452d-47fb-957b-73fc87e9483c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.519154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d918-452d-47fb-957b-73fc87e9483c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.519368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6d918-452d-47fb-957b-73fc87e9483c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.520978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 471/934] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:11.526229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61e5f6-420d-420f-9462-e5bab184ad27 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.526515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61e5f6-420d-420f-9462-e5bab184ad27 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.526808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61e5f6-420d-420f-9462-e5bab184ad27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.545758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ea61e5f6-420d-420f-9462-e5bab184ad27 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:11.547199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ea61e5f6-420d-420f-9462-e5bab184ad27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:11.581483 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.630367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70731321-31c2-4351-94b8-834a1307778b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.641216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 472/935] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:11.641619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b2dd-e61c-4248-ac31-372e1ac8f0db None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.641901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b2dd-e61c-4248-ac31-372e1ac8f0db None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.642173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b21b2dd-e61c-4248-ac31-372e1ac8f0db None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.644528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 473/936] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:11.654358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.654872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.655423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.684334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.685301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.685752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.686200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.687534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:11.688046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.689719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.694239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:11.700194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:11.700836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 464/937] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:11.706303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.733680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:11.803312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 474/938] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:11 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:11.838970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d469ae5f-8bfb-462e-8363-6d4fdcaad165 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:11.878149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:11.879317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:11.879851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:11.880308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:11.881342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] RBAC: Adding query filter params (interface=public, service_id=d16ba32cee1a465f9d63ae5e714c21c3, region_id=RegionOne) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:11.881975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:11.883793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:11.893276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 465/939] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/endpoints?service_id=d16ba32cee1a465f9d63ae5e714c21c3&interface=public®ion_id=RegionOne => generated 194 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:11.901943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:11.940796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.013123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.027919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82caf8-1d42-42a8-a741-4aa8a14801ef None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.063739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.068181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.069147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.070206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.070206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.070980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.072809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.081233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 475/940] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/services => generated 1497 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:12.085526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d916ba-5707-44d1-960c-664d70a1a39e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.130611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.132071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.132541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.133000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.144803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.146951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:12.152972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:12.154081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:12.154919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:12.163367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 466/941] 10.222.0.73 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:11 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:12.165072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.272180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92caa438-a051-4f85-be06-3ae74f30be59 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.298638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a71388-e6c6-46c9-a00c-e3d9dcc34428 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.324534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.326601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.327091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.328139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.329747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.332115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.335380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING keystone.server.flask.common [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 22:11:12.336872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.337620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.337902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.338172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.338898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:12.339201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.340609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.353005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 467/942] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:12.358092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.370551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.396476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.sql.core [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 22:11:12.396476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 22:11:12.396476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [parameters: {'id': '330181829fb84d1ba524778cb979159b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 22:11:12.396476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81612) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 330181829fb84d1ba524778cb979159b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 330181829fb84d1ba524778cb979159b. Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:11:12.408220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 23 22:11:12.409523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application yield resource Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise self.value Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application session.commit() Aug 23 22:11:12.410471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.session.flush() Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application compat.raise_( Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 22:11:12.411472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 22:11:12.412383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application result.read() Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:11:12.414205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [parameters: {'id': '330181829fb84d1ba524778cb979159b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.416314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:12.418405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 330181829fb84d1ba524778cb979159b. Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 476/943] 10.222.0.73 () {60 vars in 1140 bytes} [Tue Aug 23 22:11:12 2022] POST /identity/v3/projects => generated 262 bytes in 266 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:12.420558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.438071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.502514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14957ccf-f042-4451-92d0-7b0e37d8e6c7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.549118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c449fbc-a877-472d-b5c5-d57158601d0f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.550973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.552857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.555491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.555981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.572057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.573583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.592816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.593880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:12.594222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:12.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:12.596024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:12.596024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:12.596024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:12.596798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.597347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.602559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 468/944] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:12.606483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.614746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.617442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:12.637181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:12.638173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:12.639057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:12.645362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 477/945] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:12.648259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.692494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.746480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea5a60-c7fb-4291-9b25-d2579ca7e45b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.752167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.792557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a00c3-cdd9-4bef-835f-67e91cdf3777 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.796912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.797888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.798377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.798850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.799708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:12.800239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.802587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.820326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 469/946] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:12.829080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:12.841108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.841733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.842186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.842639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.844302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:12.844707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:12.846636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:12.877432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 478/947] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:12.948239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d466d3-0646-411c-9f0b-233bbdfcec44 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:12.969508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:12.988642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:12.989341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:12.989626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:12.989900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:12.998467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.000386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d820a945-7175-443a-8587-98476e2b1d6b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:13.013912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:13.015177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:13.016123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 470/948] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:11:12 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:13.021438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.061678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a057d-ecde-41fc-8e37-2415f1d3c321 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.062072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a057d-ecde-41fc-8e37-2415f1d3c321 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.062259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70a057d-ecde-41fc-8e37-2415f1d3c321 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.064005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 471/949] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:13.069204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.069476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.069748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.089339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:13.090560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:13.159474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e094623-d0cf-48fc-916e-28c9215ecfdd None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.190420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:13.199274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.207677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.207832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 472/950] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:13 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:13.208252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.208252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.208252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:13.208252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.208252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.227257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 479/951] 10.222.0.73 () {56 vars in 1149 bytes} [Tue Aug 23 22:11:13 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:13.232578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.355406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b90040-5c51-448c-8022-8f1b70f99db2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.365595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:13.391243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.391872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.392146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.392414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.403562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.404994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:13.420665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:13.422105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:13.423079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:13.423079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:13.423079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:13.423079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 473/952] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:13.427850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.484540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.542001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d103cffd-52ed-4e1b-b774-a855fe2fb97f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.582305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.582960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.583242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.583519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.584174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:13.584478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.585896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.599952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 480/953] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:13.606041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.620671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc02f82-f840-42cc-931a-e0c06a4316a6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.660669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:13.666380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.667134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.667508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.667873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.668641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.670346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.721392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c993a7f-5d8b-47fd-a7e7-fef499ad8363 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.737152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 474/954] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:13 2022] POST /identity/v3/services => generated 242 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:11:13.776440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.777584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.778189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.778687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.788829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.790672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:13.805409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:13.806446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:13.807351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:13.813365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 481/955] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:13.817258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:13.845428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:13.937594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf651194-5822-4bf5-b535-78b066ecdb68 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:13.976701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:13.977268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:13.977551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:13.977822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:13.978521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:13.978826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:13.980309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb871e9-a376-444e-b99e-eb9efab68934 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:13.998459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 475/956] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:14.001951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.126613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee81a3cd-b07f-4e20-892a-62ed18964fdb None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.128338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:14.173235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.174478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.174777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.175050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/da3e06d942754501b4c903e34e6bbc9d/roles/54a2522e3f374c3e8b4ff879dc696487` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.178931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=da3e06d942754501b4c903e34e6bbc9d, role_id=54a2522e3f374c3e8b4ff879dc696487)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:14.180364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:14.225769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f359178-6cc1-4464-8078-bd263d0da78e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.226353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f359178-6cc1-4464-8078-bd263d0da78e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.226353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f359178-6cc1-4464-8078-bd263d0da78e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.227939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 476/957] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:14.233270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.233558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.236105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.253965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:14.255186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:14.272797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 482/958] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:14 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/da3e06d942754501b4c903e34e6bbc9d/roles/54a2522e3f374c3e8b4ff879dc696487 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:14.334236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 477/959] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:14 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:14.626255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.737924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6fcbb-cc67-4931-94d5-1466e52702c1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.775929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.776577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.776860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.777201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.777805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:14.779187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:14.789872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:14.790990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:14.791845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 483/960] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:14.797616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:14.810073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:14.911501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982653bb-fd1a-46ee-b64e-8baa2b494629 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:14.954882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:14.955509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:14.955791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:14.956072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:14.956714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:14.957040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:14.958417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1a252c-a4b9-4a18-ab39-e658eb782475 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:14.964000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 478/961] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:14.969919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.001116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:15.052535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa748063-8ac3-4d28-b3c0-694beb74d59a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.052862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa748063-8ac3-4d28-b3c0-694beb74d59a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.053133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa748063-8ac3-4d28-b3c0-694beb74d59a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.058592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 479/962] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:15.067445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.067776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.068193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.088212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305c5876-3e85-48d9-9aa5-f6ff73590de7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:15.089042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:15.090631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:15.128221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:15.129170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.129470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.129712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.130380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:15.130688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:15.132110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:15.144199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 484/963] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:14 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:15.145698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.189557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 480/964] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:15.272385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:15.283459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cce93dd-48dd-4348-bf07-bc4dba04d960 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:15.321834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:15.322361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.322683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.322959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.323622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:15.325049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:15.410848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 485/965] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:15 2022] POST /identity/v3/endpoints => generated 349 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:15.490141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.494355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:15.626967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7e0198-52f4-4889-b5ba-fff7ad630361 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:15.670350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:15.670958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.671928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.672043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.672627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:15.674099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:15.676655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING keystone.server.flask.common [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 23 22:11:15.727490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.sql.core [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 22:11:15.727490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 22:11:15.727490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [parameters: {'user_id': 'bf95fef1935845cf920ae9c6391d3877', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 22:11:15.727490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81613) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c869f252-287e-4f2b-bf7a-8bb81427d10e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Aug 23 22:11:15.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:11:15.746510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application yield resource Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application next(self.gen) Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.rollback() Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.force_reraise() Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise self.value Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application session.commit() Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 23 22:11:15.747488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._prepare_impl() Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.session.flush() Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._flush(objects) Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application compat.raise_( Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application flush_context.execute() Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rec.execute(self) Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 23 22:11:15.748416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 23 22:11:15.749444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result = self._query(query) Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application conn.query(q) Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application result.read() Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 23 22:11:15.750418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [parameters: {'user_id': 'bf95fef1935845cf920ae9c6391d3877', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:15.751247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 23 22:11:15.753655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:15.755984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 481/966] 10.222.0.73 () {60 vars in 1131 bytes} [Tue Aug 23 22:11:15 2022] POST /identity/v3/users => generated 160 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:15.757051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.803996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555507cd-a5e8-4ec8-a417-a91e2e1cdd38 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.804546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555507cd-a5e8-4ec8-a417-a91e2e1cdd38 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.805047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555507cd-a5e8-4ec8-a417-a91e2e1cdd38 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.816442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 482/967] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:15 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:15.825072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.825920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.826581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.857103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:15.859885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:15.875639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f73d155-7781-465b-a3a2-c2609047d277 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:15.917088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:15.917705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:15.917983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:15.918255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:15.928987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:15.930545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:15.945994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:15.947946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:15.950092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 486/968] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:15 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:15.954048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:15.965644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:15.976118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.001200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 483/969] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:15 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:16.078400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dbb8f5-0dd1-4b47-8940-59c97a3485cf None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.119711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.120330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.120609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.120880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.121625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:16.121926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.123359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca485a-5bf6-4137-b8fb-937f30653241 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.162298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 487/970] 10.222.0.73 () {56 vars in 1101 bytes} [Tue Aug 23 22:11:15 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:16.292748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.425900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dc434-5561-4617-9849-267c5d7ff724 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.426174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dc434-5561-4617-9849-267c5d7ff724 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.426483 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22dc434-5561-4617-9849-267c5d7ff724 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.427804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 488/971] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:16.433463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.433801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.434016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.438380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35037255-fc26-480d-a0f3-195bd3f21aa4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.458458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:16.461266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:16.490744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.491464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.492191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.492191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.492595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.494249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:16.504163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:16.505285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:16.506141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 484/972] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:16.511448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.548949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.566416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 489/973] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:16 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:16.588045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.639158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885f7fd8-3bb0-425b-93d1-9eb950a40622 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.675255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.675874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.676156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.676431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.677093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:16.677402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.678769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.684066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 485/974] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:16.690464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.797922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aae7f-da2d-48e0-9d5e-a67d0be1231d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.832376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:16.832912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:16.833218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:16.833492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:16.834126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:16.834426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:16.835791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:16.841263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 490/975] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:16.846362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.847977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:16.915423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:16.959612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5732efb-0882-4e24-acb3-42b31859a39b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.965886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5581873-3837-4607-8b12-7c442b3d1090 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:16.984401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.000124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.000763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.001059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.001333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.009701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.010638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.011079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.011554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.012685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] RBAC: Adding query filter params (interface=public, service_id=f99bd3b1f9d544519b4ef1f1b7343b81, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:17.012845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:17.013154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.014487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:17.019581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:17.020609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:17.021504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:17.025668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 486/976] 10.222.0.73 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:17.025890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 491/977] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:11:16 2022] GET /identity/v3/endpoints?service_id=f99bd3b1f9d544519b4ef1f1b7343b81&interface=public®ion_id=RegionOne => generated 194 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:17.026740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.026838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.138576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dcfbe6-e7b8-4069-acf3-5c777d3aac85 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.144983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fd9791-e797-4c9b-a5e5-8ba7b6db53a6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.166106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.178533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.180383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.180383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.180383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.180383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:17.180670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.182068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.190068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.190699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.190968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.195907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.196089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 492/978] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:17.196176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.196176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.197038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.204794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 487/979] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/services => generated 1727 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:17.325990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7566fc3-5c52-403e-8e5a-755e62f23025 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.367127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.367777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.368058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.368332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.381384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.382902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c03f5862-0bed-459d-ad06-e9c71ebeea65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:17.399269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:17.400447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:17.401587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:17.401587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:17.401587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:17.401587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 493/980] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:17.407670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.462269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa68674-f108-4ad1-802d-51956a9bbef8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.462556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa68674-f108-4ad1-802d-51956a9bbef8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.462837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa68674-f108-4ad1-802d-51956a9bbef8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.464324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 494/981] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:17.465057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.470136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.470419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.470694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.489785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:17.490985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:17.523376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.533275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828c99c8-36e0-4fef-bb21-ab77b9f5e9f2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.570495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 495/982] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:17 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:17.573428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.574110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.574388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.574661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.575309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:17.575613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.576979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.593392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 488/983] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:17.596666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.707605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce0afff-695d-4efc-94ee-ee1688cb3db5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.746823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.747463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.747747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.748022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.755496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.756991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:17.769843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:17.770847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 23 22:11:17.771847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:17.773336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 496/984] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:17.776721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.847065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:17.880926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.889277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a082732-c250-4524-a342-2d8cbbec82c3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.893619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:17.929321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:17.929939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:17.930216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:17.930492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:17.931235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:17.931586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:17.933457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:17.960852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 489/985] 10.222.0.73 () {56 vars in 1149 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:17.965768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cbfe30-0fb8-4685-a791-4b7b8a015f53 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:17.967279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.004463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.005145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.005427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.005714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.006299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.007700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:18.017969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:18.019017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:18.022259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 497/986] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:18.025870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.089553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885642ba-f5c9-47d0-9e20-1c959b4b5dff None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.128145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.128771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.129064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.129338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.136766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.148387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909fb824-f307-46d2-806e-4844de409ae0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.164576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.189667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.190288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.195656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.195656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.195656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:18.195656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.195656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.198684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 498/987] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:18.205461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.213135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.213915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.321038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41bd394-034d-457d-aadf-53f42cc3b71c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.335710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 490/988] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:17 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:18.341587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.372409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.373048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.373336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.373611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.374298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:18.374606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.375989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.381262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 499/989] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:18.387419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.410206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.417638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.458595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a9d8a9-e1fa-4429-8d7f-058acc6758f7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.499726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.500319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebc0065-57ee-4814-9d60-9e919847ff30 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.500635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.500925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.501239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.512093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.513584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:18.529965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:18.531088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:18.531975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:18.531975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:18.531975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:18.531975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 491/990] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:18.537161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.541712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.542330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.542609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.542882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.543593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] RBAC: Adding query filter params (interface=admin, service_id=d16ba32cee1a465f9d63ae5e714c21c3, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:18.543903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.545316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.556913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 500/991] 10.222.0.73 () {56 vars in 1249 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/endpoints?service_id=d16ba32cee1a465f9d63ae5e714c21c3&interface=admin®ion_id=RegionOne => generated 193 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:18.557888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.646357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.659150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e272fe57-3ebe-4f8e-b72c-8ba7525ad8a7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.665175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b87ed6-6165-4109-bfd6-8af1a0d3688a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.674413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:18.696027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.696635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.696912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.697419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.697906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:18.698204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.699576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.704619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.705277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.705552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.705852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.706522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.707952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.713549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 492/992] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:18.719690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:18.733039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 501/993] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:18.852763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015d636f-db5d-4095-a4b0-9d9d96c042e4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:18.901297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:18.901964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:18.902242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:18.902585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:18.911253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:18.912775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:18.924975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:18.925984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:18.926879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:18.929099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 493/994] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:18.932239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:19.040976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8423bb56-ed21-4d2f-b9ba-bf13d68ecf11 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:19.078847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:19.079838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:19.080331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:19.080796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:19.081701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:19.082187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:19.083800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1238c-e500-415c-b7ae-1e4e647c0ab6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:19.096821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 502/995] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:19.105414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:19.142336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:19.190708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3f133-1b4d-408c-bd5c-da46907dd7a5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:19.191307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3f133-1b4d-408c-bd5c-da46907dd7a5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:19.191788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d3f133-1b4d-408c-bd5c-da46907dd7a5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:19.193728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 503/996] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:19.198960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835f4af-683f-4718-972e-028e69ade76d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:19.199239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835f4af-683f-4718-972e-028e69ade76d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:19.199680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a835f4af-683f-4718-972e-028e69ade76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:19.213820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:19.219506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a835f4af-683f-4718-972e-028e69ade76d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:19.220310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbcd727-8dfe-46e5-8089-1d8cb07b1f77 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:19.220884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a835f4af-683f-4718-972e-028e69ade76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:19.259500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:19.260107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:19.260388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:19.260660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:19.261424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] RBAC: Adding query filter params (role.id=54a2522e3f374c3e8b4ff879dc696487, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=da3e06d942754501b4c903e34e6bbc9d) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:19.261731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:19.263188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:19.281041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 494/997] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:19 2022] GET /identity/v3/role_assignments?user.id=da3e06d942754501b4c903e34e6bbc9d&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=54a2522e3f374c3e8b4ff879dc696487 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:19.313214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 504/998] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:19 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:19.595492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:19.715427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:19.720778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138c35cc-c55c-4650-b193-6ed28c551b16 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:19.756851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:19.757521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:19.757811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:19.758089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:19.758676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:19.760090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:19.770178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:19.771228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:19.772066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 495/999] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:19 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:19.777538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835f4af-683f-4718-972e-028e69ade76d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:19.898154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a835f4af-683f-4718-972e-028e69ade76d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:19.938267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:19.938918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:19.939266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:19.939459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:19.940067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:19.940365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:19.941990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:19.944239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:19.948690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 505/1000] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:19 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:19.955426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:20.071947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99fc8e8-868f-46f3-a905-ce1dcd120393 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:20.111837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:20.112871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.113414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.113943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.114816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] RBAC: Adding query filter params (type=compute) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:20.115322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:20.116927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4e742a-a918-4176-bc5b-17b84a58f27e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:20.123331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 496/1001] 10.222.0.73 () {56 vars in 1112 bytes} [Tue Aug 23 22:11:19 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:20.129368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:20.237147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbff97a-d90d-49b3-83ce-9251e3032e14 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:20.273903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30963c2-a61e-4766-bbc4-140c5148f036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:20.274522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30963c2-a61e-4766-bbc4-140c5148f036 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.274804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30963c2-a61e-4766-bbc4-140c5148f036 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.275136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30963c2-a61e-4766-bbc4-140c5148f036 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.275822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30963c2-a61e-4766-bbc4-140c5148f036 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:20.277228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30963c2-a61e-4766-bbc4-140c5148f036 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:20.381056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 506/1002] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:11:20.716162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8bc14-0fd5-4db0-a83d-a81d2a0ad6a2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.716681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8bc14-0fd5-4db0-a83d-a81d2a0ad6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.716758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c8bc14-0fd5-4db0-a83d-a81d2a0ad6a2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.718843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 497/1003] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:20.724373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:20.724662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:20.724940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:20.743827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:20.745094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:20.798525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:20.822140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 507/1004] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:20 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:21.097671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8bc14-0fd5-4db0-a83d-a81d2a0ad6a2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.128733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.210960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c8bc14-0fd5-4db0-a83d-a81d2a0ad6a2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.247845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.248555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.248834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.249355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.249941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.251349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5e9e8128-b284-4540-9e48-af3e032e4b36 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:21.260406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:21.261627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:21.266181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 498/1005] 10.222.0.73 () {56 vars in 1126 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:21.268526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.272259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7fdd7-f7f4-42c7-9572-e674bdd9a814 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.272545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7fdd7-f7f4-42c7-9572-e674bdd9a814 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.272816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e7fdd7-f7f4-42c7-9572-e674bdd9a814 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.274206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 499/1006] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:21.279221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.279501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.279896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.301214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:21.302644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:21.373243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd0d4f6-74c5-46d2-860f-6d3d4d3c66f2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.375861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 500/1007] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:21 2022] POST /identity/v3/auth/tokens => generated 1726 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:21.410808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.411810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.412315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.412801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.413665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:21.414172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.415750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.422123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 508/1008] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:21.428293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.461846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.559265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e492d8-54f9-488a-8901-9f38edd55ac3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.598059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.598627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.598907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.599181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.599853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:21.600159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.601573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.607170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 501/1009] 10.222.0.73 () {56 vars in 1122 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:21.613590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.651570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.673624 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.743879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a90b16-0ef5-4912-bcdc-11920cd17d4e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.786491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:21.789565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.790184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.790465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.790737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.791418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.793392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.810786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d408463-f985-4ba0-b8f9-18b9068a9a15 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:21.851384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:21.852139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:21.852429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:21.852701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:21.853346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:21.854753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:21.866467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:21.869064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:21.869995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 502/1010] 10.222.0.73 () {56 vars in 1105 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/services/image => generated 86 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:21.878430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:21.901047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 509/1011] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:21 2022] POST /identity/v3/endpoints => generated 325 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:22.007745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e172f-3a23-4af5-bcdb-76b7193b2e90 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:22.067319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:22.067939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:22.068214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:22.068527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:22.069230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] RBAC: Adding query filter params (name=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:22.069574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:22.072245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c090efc-d465-4963-b9ba-5d614f75dc4e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:22.080307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 503/1012] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:22.086368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:22.200789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d843462b-5473-4a9a-9bdb-52689237e118 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:22.209228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:22.236520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:22.237319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:22.237604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:22.237869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:22.238509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] RBAC: Adding query filter params (type=image) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:22.238811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:22.240175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c814a5-7980-4110-baf5-8f5b84208a50 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:22.253099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 510/1013] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:22 2022] GET /identity/v3/services?type=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:22.451309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bac6a-baf4-4b3f-8949-8c9ae0fd9d7f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:22.451691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bac6a-baf4-4b3f-8949-8c9ae0fd9d7f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:22.452453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89bac6a-baf4-4b3f-8949-8c9ae0fd9d7f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:22.454361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 504/1014] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:22.460153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:22.460672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:22.460672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:22.481119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:22.482356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:22.566101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 511/1015] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:22 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:22.835100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bac6a-baf4-4b3f-8949-8c9ae0fd9d7f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:22.847168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:22.944204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89bac6a-baf4-4b3f-8949-8c9ae0fd9d7f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:22.979836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:22.980530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:22.980817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:22.981115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:22.993046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:22.997793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:22.998766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:23.000669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:23.007564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 505/1016] 10.222.0.73 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:22 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:23.007724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:23.109065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:23.112860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee868bd-c222-497b-a02f-dfa3183bc0a2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:23.147990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:23.148648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.148904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.149528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.150385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:23.150687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:23.152472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:23.167444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 512/1017] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:23.169334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:23.248121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:23.288284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bcceae-fc78-4d76-81b6-007d5ebaba17 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:23.326334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:23.327618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.328012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.328341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.340972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:23.342612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:23.359650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:23.360627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:23.361836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:23.361836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:23.361836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:23.361836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 506/1018] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:23.367048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:23.480825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775492cd-9c81-4815-9795-ab4eb1511d38 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:23.518705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:23.519321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.519601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.519873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.520522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:23.520824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:23.522222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:23.537872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 513/1019] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:23.543530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:23.656547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6874c53-29a3-4e89-92cc-8a98b077662c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:23.677281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:23.692830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:23.693511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.697465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.697465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.702021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:23.704384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:23.716213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:23.718049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:23.719001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 507/1020] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:11:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:23.724151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:23.750959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:23.837354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b250e43a-4569-4086-abe5-bbe5d854cf7d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:23.876789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:23.877444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.877727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.878032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.878746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:23.879054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:23.880618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954f5c79-d396-4b8e-902f-0f233a9cf9d4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:23.900742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 514/1021] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:11:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:23.907820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:23.962340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d03d-daa0-48fd-a50d-bdb26fe47138 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.962680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d03d-daa0-48fd-a50d-bdb26fe47138 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.962891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6785d03d-daa0-48fd-a50d-bdb26fe47138 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.964251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 515/1022] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:23.970758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc104-73eb-405f-a70e-8a4816f26587 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:23.971341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc104-73eb-405f-a70e-8a4816f26587 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:23.971615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6efc104-73eb-405f-a70e-8a4816f26587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:23.989521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d6efc104-73eb-405f-a70e-8a4816f26587 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:23.990727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d6efc104-73eb-405f-a70e-8a4816f26587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:24.023650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f1a15d-d958-4766-affe-a0f53bca79a8 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.063784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.064438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.065217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.065301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 516/1023] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:23 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:24.065377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.126347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.146338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.147775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.200781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360cd4-6d86-4692-8e30-847e0045cd88 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.201154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360cd4-6d86-4692-8e30-847e0045cd88 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.201401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b360cd4-6d86-4692-8e30-847e0045cd88 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.203539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 517/1024] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:24.208627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.227655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.227655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.227655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:24.228250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:24.231189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.254606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 508/1025] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:23 2022] GET /identity/v3/auth/tokens => generated 1873 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:24.260847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.303218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 518/1026] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:24 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:24.342192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.347965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.368747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d80b701-5835-4207-bc61-9f61938f543a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.406993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.407874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.408191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.408427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.419206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.420670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:24.436148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:24.437679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:24.438633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:24.438633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:24.438633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:24.443022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 509/1027] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:24.443965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.453134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59cf05e-5b13-4d54-b6f6-57e10f9c72ff None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.491097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.491856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.492021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.492288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.493461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.494903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.496665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 519/1028] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:24.503074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.513676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.566987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6401bb1a-cf46-4296-a209-1c0c9c97b68e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.599676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.605173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.605790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.606119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.606382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.607024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:24.607325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.609584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.613821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45df90-d147-44e1-adb1-f10821c8c6a4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.621249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 510/1029] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:24.622046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.649079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.649686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.649963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.650228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.650864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.652309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.728327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c8a34e-cd2c-4816-a496-1ba2b28d16f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.763861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.764598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.764881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.765210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.765911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.767308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.791594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 520/1030] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:24 2022] POST /identity/v3/projects => generated 311 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:11:24.792453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:24.861062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 511/1031] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:24 2022] POST /identity/v3/services => generated 242 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:24.868984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:24.921969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b60d48-2eac-48a4-b561-ba1e0dc6d738 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:24.972917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:24.973605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:24.973883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:24.974154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:24.984310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:24.984555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:24.998228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:24.999943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:25.001151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 521/1032] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:25.007054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:25.068198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:25.124202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa01d93e-bd3e-4d0c-b274-58bd795547a2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:25.160071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:25.160971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:25.161426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:25.161870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:25.162741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:25.163196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:25.164780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c296513a-5bfc-4417-a782-97cf7275f0bc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:25.178298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 512/1033] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:25.184040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:25.293672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c604e35f-1da1-42cf-aff3-5f1f0bf0f63a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:25.312047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:25.332214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:25.332828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:25.333136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:25.333412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:25.334140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] RBAC: Adding query filter params (role.id=54a2522e3f374c3e8b4ff879dc696487, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=059f72c1793e492fb052bebbf29b5bec) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:25.334444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:25.335868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2196d449-ce82-45e9-8a1d-b0f287d7e686 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:25.349053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 522/1034] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:25 2022] GET /identity/v3/role_assignments?user.id=059f72c1793e492fb052bebbf29b5bec&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=54a2522e3f374c3e8b4ff879dc696487 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:26.806426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8cb90-4e36-4561-83bd-485261e73d92 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:26.807108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8cb90-4e36-4561-83bd-485261e73d92 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:26.807108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8cb90-4e36-4561-83bd-485261e73d92 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:26.809053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 513/1035] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:26.814355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:26.814833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:26.814909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:26.834137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:26.835332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:26.901109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16984321-7616-4d9b-8a9b-93950a8996d4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:26.901479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16984321-7616-4d9b-8a9b-93950a8996d4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:26.901678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16984321-7616-4d9b-8a9b-93950a8996d4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:26.903188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 514/1036] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:26.909517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:26.909796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:26.910070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:26.915958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 523/1037] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:26.928958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:26.930190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:27.009173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 515/1038] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:26 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:27.190370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.282654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.300983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf3ec9-ceda-4f1e-82ea-3a0a70d3813a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:27.313075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:27.329050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:27.339101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:27.339738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.340013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.340280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.341404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:27.342965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:27.344567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 524/1039] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:27.350391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.393447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e440b6-1f79-433e-b4ed-d4f06638aa0b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:27.428642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:27.429312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.429587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.430274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.430866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:27.432223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f57e4ea3-2e1a-4021-ba7b-eecfe26a9cf1 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:27.441888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:27.442937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:27.443790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 516/1040] 10.222.0.73 () {56 vars in 1105 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/services/image => generated 86 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:27.444593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c12a1f-dcc5-4c0f-a9d0-415eba872051 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.444898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c12a1f-dcc5-4c0f-a9d0-415eba872051 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.444990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c12a1f-dcc5-4c0f-a9d0-415eba872051 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.446417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 517/1041] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:27.448861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c12a1f-dcc5-4c0f-a9d0-415eba872051 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.459827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b7458-0ec4-4a47-bf0c-62b4a17b30f5 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:27.495735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334221d1-2fb6-4c24-8dcd-3e7467496444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:27.496350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334221d1-2fb6-4c24-8dcd-3e7467496444 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.496622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334221d1-2fb6-4c24-8dcd-3e7467496444 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.496892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334221d1-2fb6-4c24-8dcd-3e7467496444 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.497576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334221d1-2fb6-4c24-8dcd-3e7467496444 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:27.499489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334221d1-2fb6-4c24-8dcd-3e7467496444 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:27.558090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c12a1f-dcc5-4c0f-a9d0-415eba872051 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:27.566045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:27.593701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:27.594312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.594592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.594869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.595561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] RBAC: Adding query filter params (name=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:27.595861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:27.596245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 525/1042] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:27 2022] POST /identity/v3/users => generated 285 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:27.597265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:27.597593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.597952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.598159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.602690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 518/1043] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:27.610057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.629601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:27.630853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:27.661694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:27.721632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 526/1044] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:27 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:27.745810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00be70-3da8-417c-920d-61244a1519c2 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:27.786157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:27.787084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.787365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.787645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.788345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:27.788648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:27.790048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:27.796131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 519/1045] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:27.802079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.910490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc55728-6d4e-4c9a-b5f8-08cfc65627c8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:27.945677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:27.946295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:27.946574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:27.946847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:27.947558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] RBAC: Adding query filter params (interface=public, service_id=343468dc7f074c51bf4f11f62506b3a2, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:27.947973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:27.949276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:27.955484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 527/1046] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/endpoints?service_id=343468dc7f074c51bf4f11f62506b3a2&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:27.960969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:27.986111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.002925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.021936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.085953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abbca37e-c5c6-4f1a-8e1d-b844d9d29de4 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.103399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434724e-045f-47ee-8d14-ccd0d7fd9048 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.124770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.125492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.125807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.126556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.127184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.128717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.136115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 520/1047] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/services => generated 1957 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:11:28.141436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.142238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.142588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.142928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.157816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:28.163063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:28.164103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:28.166176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 528/1048] 10.222.0.73 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:27 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:28.170730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.281680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.285695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6a6617-583c-4bd1-8af4-1ed826cd0ea3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.322082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.322698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.322976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.323243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.323908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:28.324204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.325603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.334574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 521/1049] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:28.340651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.433902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.447119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3dbd54-8dc1-4538-9c04-7bc78e29c8d7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.481867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.482501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.482777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.483564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.500640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.502124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:28.518175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:28.519189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:28.520347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:28.520347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:28.520347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:28.524025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 529/1050] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:28.525426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.635806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02551ccc-b998-4af8-b665-73cb127775d9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.672316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.672918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.673222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.673492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.674128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:28.674423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.675779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.689133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 522/1051] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:28.694895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.810501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2246a42-1587-433b-bb3e-030f615306d2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:28.849739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:28.850733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:28.851221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:28.851702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:28.860306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:28.862091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:28.865050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:28.875523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:28.876807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:28.882592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 530/1052] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:28.884762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:28.908720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:28.991957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7347e49-18b1-4d3c-8459-9ac766d008ba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.032918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.033580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.033917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.034197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.035005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:29.035315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.036751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:29.059970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 523/1053] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:11:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:29.061909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:29.168932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5636ada1-e6c5-4d60-9c73-5e0b35a0111c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.206874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.207516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.207877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.213124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.220351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.221936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:29.237351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:29.238481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:29.248351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:29.248351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:29.248351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:29.248351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 531/1054] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:29.249146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:29.332708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:29.354956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24141f-2564-452d-8b33-6860807f3f8f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.391724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.392331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.392615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.392890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.393564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:29.393869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.395263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:29.413255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 524/1055] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:29.414464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:29.472678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:29.527553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258191b1-bbc7-4916-bd85-8acd3128c670 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.568896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.569645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.569901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.570122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.579395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.580871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9ef33a4b-7f93-43ae-b4ee-88e3016b5af4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:29.593434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:29.594431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:29.595876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:29.597377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 532/1056] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:29.600265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:29.677804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78107bd-e9c2-4046-a668-2cfea3062346 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.678155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78107bd-e9c2-4046-a668-2cfea3062346 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.678364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78107bd-e9c2-4046-a668-2cfea3062346 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.679921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 533/1057] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:29.685578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.685854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.686650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.707401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:29.708617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:29.726749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb01f7b-cb4e-4d9a-b652-3c998502743d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.770036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.770967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.771250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.771527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.772250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:29.772558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.774095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:29.782575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:29.788116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 525/1058] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:29.792376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 534/1059] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:29 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:29.797716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:29.912089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20dda0c-bc94-4734-af9c-09b1bc9ba0db None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:29.953324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0e38db-6f4c-444e-8ca6-135034801773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:29.954428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e38db-6f4c-444e-8ca6-135034801773 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:29.954885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e38db-6f4c-444e-8ca6-135034801773 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:29.955329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0e38db-6f4c-444e-8ca6-135034801773 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/059f72c1793e492fb052bebbf29b5bec/roles/54a2522e3f374c3e8b4ff879dc696487` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:29.957827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e38db-6f4c-444e-8ca6-135034801773 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=059f72c1793e492fb052bebbf29b5bec, role_id=54a2522e3f374c3e8b4ff879dc696487)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:29.959477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0e38db-6f4c-444e-8ca6-135034801773 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:30.059390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 526/1060] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:29 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/059f72c1793e492fb052bebbf29b5bec/roles/54a2522e3f374c3e8b4ff879dc696487 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:30.075928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.124772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ee208-e2c8-443f-9976-e5d18b669fa8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.125166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ee208-e2c8-443f-9976-e5d18b669fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.125354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ee208-e2c8-443f-9976-e5d18b669fa8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.126729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 527/1061] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:30.130542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:30.133149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.133436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.133740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.153529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:30.155418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:30.196253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18433bb8-b36e-4686-8baa-2a5fe8b4fbcd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.229604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:30.241872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:30.242514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.242796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.243070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.248693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 528/1062] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:30 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:30.255481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:30.260459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:30.261395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:30.262675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 535/1063] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:30.268217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.376623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2ea17-6c4a-4096-b2dd-a5ecaaa1cd66 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.412949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:30.413592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.413870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.414141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.414806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:30.415107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:30.416586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:30.430651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 529/1064] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:30.432251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.518233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.550486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bfe3c3-7ad4-4a38-8f84-92246b98b6c2 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.554630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:30.578027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:30.590818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:30.591816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.592335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.592954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] PATH_INFO: `/v3/users/2161b0ea41334c86ac4a7cb654781789` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.608514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] RBAC: Authorizing `identity:get_user(user_id=2161b0ea41334c86ac4a7cb654781789)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:30.610690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:30.612906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 536/1065] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/users/2161b0ea41334c86ac4a7cb654781789 => generated 285 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:30.619191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.639569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18477790-513c-47a1-9577-cf2f4cca76b1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.676523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:30.677491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.677491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.677796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] PATH_INFO: `/v3/services/image` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.678274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:30.679627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:30.690890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:30.692027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:30.692887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 530/1066] 10.222.0.73 () {56 vars in 1105 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:30.697994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.748729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac42099-c4bc-4d85-83c0-359dec64010b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.785182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:30.785706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.785979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.786248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.815352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d0d75b-951b-4fcd-8b28-029b3387a267 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.818184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:30.832043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:30.854364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:30.854979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:30.855262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:30.855537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:30.856171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] RBAC: Adding query filter params (name=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:30.856475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:30.858549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:30.864133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:30.864287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 531/1067] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:30.865370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:30.869943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:30.978180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cce611-a3a3-49b1-aa4f-fedecff9dc0d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:30.989533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 537/1068] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:11:30.991319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:31.016244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:31.017036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.017330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.017603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.018246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] RBAC: Adding query filter params (type=image) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:31.018551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:31.020067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:31.030653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 532/1069] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:31.032053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:31.057436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:31.079643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:31.101400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a04f9af-5bec-4767-9af9-6b1e4e384417 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:31.142228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:31.142868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.143151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.143453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.144569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] RBAC: Authorizing `identity:get_project(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:31.146034 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:31.147670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 538/1070] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:30 2022] GET /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:31.150283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304efd66-9528-4e60-9ea0-949e8202f35c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:31.155145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:31.190435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a93b41-e50e-42cf-9ae2-fd65a01d7786 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:31.191041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a93b41-e50e-42cf-9ae2-fd65a01d7786 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.191320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a93b41-e50e-42cf-9ae2-fd65a01d7786 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.191594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a93b41-e50e-42cf-9ae2-fd65a01d7786 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.192460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a93b41-e50e-42cf-9ae2-fd65a01d7786 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:31.193727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a93b41-e50e-42cf-9ae2-fd65a01d7786 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:31.265373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b16b00-25e0-4d02-aa86-ed6a4922ab6b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:31.288560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 533/1071] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:11:31 2022] POST /identity/v3/endpoints => generated 327 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:31.305365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:31.322654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:31.323194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:31.323472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:31.323739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:31.324775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe, user.id=2161b0ea41334c86ac4a7cb654781789) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:31.325056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:31.327077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-606726e8-f321-42d8-8b04-6a04847bbe4e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:31.339569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 539/1072] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:31 2022] GET /identity/v3/role_assignments?user.id=2161b0ea41334c86ac4a7cb654781789&scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:32.063645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0c914-57f9-442a-8d35-7e6d6e2c3ed8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:32.064005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0c914-57f9-442a-8d35-7e6d6e2c3ed8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:32.064188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f0c914-57f9-442a-8d35-7e6d6e2c3ed8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:32.065729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 534/1073] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:32.071081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:32.071371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:32.071645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:32.089846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:32.091050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:32.171312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 540/1074] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:32.447722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0c914-57f9-442a-8d35-7e6d6e2c3ed8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:32.560630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:32.566351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f0c914-57f9-442a-8d35-7e6d6e2c3ed8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:32.601045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:32.601684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:32.601957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:32.602224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:32.612779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:32.617801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:32.618920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 23 22:11:32.619972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:32.622683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 535/1075] 10.222.0.73 () {56 vars in 1120 bytes} [Tue Aug 23 22:11:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:32.629079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:32.734845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502d0bd3-cb97-41ff-a815-421cfed766d1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:32.754998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:32.773220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:32.773837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:32.774117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:32.774390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:32.775062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:32.775366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:32.776771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:32.789973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 541/1076] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:11:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:32.792077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:32.903522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606a89c-bd8b-43f2-8b56-ebfd50f24129 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:32.945685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:32.946355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:32.946634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:32.946909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:32.959910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:32.961417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:32.977449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:32.978498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:32.979403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:32.979403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:32.979403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:32.979403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 536/1077] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:32.985785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:33.106516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba89b611-459b-40d1-a235-ad4012987f23 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:33.143646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:33.144273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.144553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.144825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.145524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:33.145833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:33.147225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b4b70d-b5be-4f47-9014-5bef6d507f3f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:33.161595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 542/1078] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:33.167287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:33.182103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:33.275704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81abd3df-7ff0-4af9-ba8a-0c5ce36dc43c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:33.314746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:33.315383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.315660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.315931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.324407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:33.326123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:33.327339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bb07b-7256-4c8a-b9e8-0a751213f974 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.327783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bb07b-7256-4c8a-b9e8-0a751213f974 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.327977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33bb07b-7256-4c8a-b9e8-0a751213f974 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.329380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 543/1079] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:11:33.334737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173774a8-a8d1-4339-a8b0-da032d6df285 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.335025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173774a8-a8d1-4339-a8b0-da032d6df285 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.335300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-173774a8-a8d1-4339-a8b0-da032d6df285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6815cb71-ef3d-4ea3-8d1e-0b3179fbe45f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:11:33.339271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:33.340402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:33.341465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 537/1080] 10.222.0.73 () {56 vars in 1093 bytes} [Tue Aug 23 22:11:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:11:33.343020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862e078-8b59-4963-8668-efb420fe507a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.344267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862e078-8b59-4963-8668-efb420fe507a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.344560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862e078-8b59-4963-8668-efb420fe507a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.345972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 538/1081] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:33.346851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862e078-8b59-4963-8668-efb420fe507a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:33.354774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-173774a8-a8d1-4339-a8b0-da032d6df285 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:33.355975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-173774a8-a8d1-4339-a8b0-da032d6df285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:33.393194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:33.419232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:33.427360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 544/1082] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:33.428948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.429553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.430079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.450512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:33.451941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:33.458910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862e078-8b59-4963-8668-efb420fe507a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:33.500437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:33.501081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.501364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.501640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.502390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:33.502693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:33.504118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:33.523984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 539/1083] 10.222.0.73 () {56 vars in 1133 bytes} [Tue Aug 23 22:11:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:33.529932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:33.537045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 545/1084] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:33.642104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb4a00-d91b-4d0d-a151-cd779e0fac92 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:33.669579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:33.679064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:33.679672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.679951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.680355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.700140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:33.762656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:33.764144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:33.809782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:33.814516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057701b2-aa0e-4ed3-84df-d14b6cf9bca3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:33.855320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:33.855943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:33.856224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:33.856523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:33.872468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 540/1085] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:33.874588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:33.902254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:33.936799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:33.938311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:33.981803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365523ba-7997-4ec4-93ba-6b640c797b44 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.019130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.019761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.020042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.020321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.031410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:34.036528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:34.038251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 541/1086] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:34.039518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.044574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.048798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 546/1087] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:33 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 23 22:11:34.049688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.131437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.144449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968deab-8592-4fbb-972d-1ee908d11663 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.154390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35445d11-53cd-413e-83f3-a4b14ea91636 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.178144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.178771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.179050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.179323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.190533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.190892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.191505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.191784 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.191980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.192081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.192701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:34.193000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.194434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.203294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 547/1088] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:34.204120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:34.208945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:34.212346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:34.213302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:34.213302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:34.213302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:34.213302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 542/1089] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:34.213302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.267610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.306388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f94d27-7cac-4ca7-ae9f-2dc441f2cec6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.317171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572df1be-a176-4f77-8a32-eb074127161a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.341445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.342154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.342431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.342711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.351764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.352393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.352671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.352939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] PATH_INFO: `/v3/users/2161b0ea41334c86ac4a7cb654781789` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.353319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.354099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] RBAC: Authorizing `identity:get_user(user_id=2161b0ea41334c86ac4a7cb654781789)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.354849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.355490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.357092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 543/1090] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/users/2161b0ea41334c86ac4a7cb654781789 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:34.358356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.369602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:11:34.371902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:34.372907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:11:34.374665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:11:34.374665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:11:34.374665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:34.374665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 548/1091] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:34.374665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.466413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2dd683-543c-4654-a919-f73aaf7e3c31 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.474868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7402bf83-153a-4d8e-b531-44ed8148df1d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.489801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.506067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.507964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.507964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.507964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.508330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:34.508330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.509729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.513181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.513982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.514310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.514595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.515816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] RBAC: Authorizing `identity:get_project(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.517333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.519368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 549/1092] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:34.520201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.528575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 544/1093] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:34.529480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.600243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.629412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d88a3a8-98ef-47fd-a81d-07e01ff84839 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.634419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620b92f-244d-461d-96ce-95824054a51a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.664332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.664947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.665256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.665541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.666190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:34.666498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.667869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.670855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.671525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.671804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.672082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe/users/2161b0ea41334c86ac4a7cb654781789/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.674096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] RBAC: Authorizing `identity:create_grant(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe, user_id=2161b0ea41334c86ac4a7cb654781789, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.675506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.680543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 550/1094] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:34.681416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:34.735270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:34.756879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 545/1095] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:34 2022] PUT /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe/users/2161b0ea41334c86ac4a7cb654781789/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:34.804793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673ef17-9f67-4a07-8329-6a49ffff753b None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:34.853533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:34.854202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:34.854484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:34.854764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:34.863256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:34.864763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:11:34.877537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:34.878504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:11:34.879478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 551/1096] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:34.886544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:35.003845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fce002-9b0f-44ca-84df-ff8522dca487 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:35.033900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:35.041823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:35.050222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:35.050222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:35.050222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:35.050222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:35.050222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:35.050222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f30eb-4fa1-4ecd-873d-c748a2f63fac None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:35.058173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 546/1097] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:35.065095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:35.168544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1efbfc-3147-4d22-b924-f28a7d11be38 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:35.170659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:35.205275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:35.205859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:35.206354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:35.206824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:35.207847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] RBAC: Adding query filter params (role.id=54a2522e3f374c3e8b4ff879dc696487, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=059f72c1793e492fb052bebbf29b5bec) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:35.208156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:35.209619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22ef391-dd78-49c0-9302-1ea3146c2116 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:35.215759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 552/1098] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:35 2022] GET /identity/v3/role_assignments?user.id=059f72c1793e492fb052bebbf29b5bec&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=54a2522e3f374c3e8b4ff879dc696487 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:36.731597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4118e-250f-44a7-a909-689c40ed3d22 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:36.732109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4118e-250f-44a7-a909-689c40ed3d22 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:36.732307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b4118e-250f-44a7-a909-689c40ed3d22 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:36.734516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 547/1099] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:36.739862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:36.740280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:36.740413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:36.758970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:36.760165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:36.828517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 553/1100] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:37.101633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4118e-250f-44a7-a909-689c40ed3d22 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.204722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b4118e-250f-44a7-a909-689c40ed3d22 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.238739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.239363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.239646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.239912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.251751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:37.256681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:37.257814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:37.258766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 548/1101] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:37.264555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.356682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.363066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664cf82-daa8-4d32-808f-73e0fe5e9f5b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.396785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.397425 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.397706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.397980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.398644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:37.398945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:37.400469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:37.409444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 554/1102] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:37.415571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.466166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.519282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0bc360-ecba-4174-a074-7210d94353f1 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.552716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.553463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.553745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.554019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] PATH_INFO: `/v3/users/2161b0ea41334c86ac4a7cb654781789` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.555158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] RBAC: Authorizing `identity:get_user(user_id=2161b0ea41334c86ac4a7cb654781789)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:37.556584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:37.558221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 549/1103] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/users/2161b0ea41334c86ac4a7cb654781789 => generated 285 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:37.565125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.666401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5333e4-a4a5-4841-a966-ae9220cadd31 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:37.700887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:37.701836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:37.702176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:37.702473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:37.735871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:37.775605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:37.777204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:37.878219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 555/1104] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:37.884250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:37.983730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a5eae7-2822-43e0-baa9-de3fae92f279 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:38.000953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:38.018455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f14ef2f-b9cb-4cd8-af22-aeddb3235105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:38.019084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ef2f-b9cb-4cd8-af22-aeddb3235105 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:38.019360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ef2f-b9cb-4cd8-af22-aeddb3235105 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:38.019633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f14ef2f-b9cb-4cd8-af22-aeddb3235105 None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:38.020741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14ef2f-b9cb-4cd8-af22-aeddb3235105 None admin] RBAC: Authorizing `identity:get_project(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:38.022196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f14ef2f-b9cb-4cd8-af22-aeddb3235105 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:38.023829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 550/1105] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:37 2022] GET /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:38.030105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:38.088806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:38.130130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3241e5f7-2613-4d07-a679-ab23df88ed8d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:38.163617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:38.164225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:38.164506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:38.164779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:38.165606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe, user.id=2161b0ea41334c86ac4a7cb654781789) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:38.165911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:38.167328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ee8535-c838-4f74-8928-bb04909934f0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:38.173927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 556/1106] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:38 2022] GET /identity/v3/role_assignments?user.id=2161b0ea41334c86ac4a7cb654781789&scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:40.116285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60534d4b-68c4-43aa-bdbc-8b697498d2bf None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.116840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60534d4b-68c4-43aa-bdbc-8b697498d2bf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.116840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60534d4b-68c4-43aa-bdbc-8b697498d2bf None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.118181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 551/1107] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:40.124634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.125311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.125311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.143801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:40.144997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:40.212084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 557/1108] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:40.481535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60534d4b-68c4-43aa-bdbc-8b697498d2bf None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:40.587852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60534d4b-68c4-43aa-bdbc-8b697498d2bf None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:40.621831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7287a8-217f-44c4-864d-54105c71cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:40.622453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7287a8-217f-44c4-864d-54105c71cc6c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.622732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7287a8-217f-44c4-864d-54105c71cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.623617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7287a8-217f-44c4-864d-54105c71cc6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.624726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7287a8-217f-44c4-864d-54105c71cc6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:40.626181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f7287a8-217f-44c4-864d-54105c71cc6c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:40.632901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 552/1109] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:40.633751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:40.685730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:40.731386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6058a4-baaf-46ee-8d90-f8e84d21fab8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:40.766077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd53ad5c-6d77-44dc-940d-eece08c5271a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:40.766663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ad5c-6d77-44dc-940d-eece08c5271a None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:40.766969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ad5c-6d77-44dc-940d-eece08c5271a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:40.767246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd53ad5c-6d77-44dc-940d-eece08c5271a None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:40.767907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53ad5c-6d77-44dc-940d-eece08c5271a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:40.769367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd53ad5c-6d77-44dc-940d-eece08c5271a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:40.915430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 558/1110] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:40 2022] POST /identity/v3/users => generated 286 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:42.913944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdd0d8-4a81-4788-a136-7a57f0587353 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:42.914471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdd0d8-4a81-4788-a136-7a57f0587353 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:42.914471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdd0d8-4a81-4788-a136-7a57f0587353 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:42.915723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 553/1111] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:42.922030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:42.922542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:42.922630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:42.942669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:42.943923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:43.023227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 559/1112] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 23 22:11:43.292869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdd0d8-4a81-4788-a136-7a57f0587353 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:43.369335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:43.418286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdd0d8-4a81-4788-a136-7a57f0587353 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:43.452832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:43.453492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:43.453774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:43.454047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] PATH_INFO: `/v3/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:43.455441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] RBAC: Authorizing `identity:get_role(role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:43.456833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:43.458190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 554/1113] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:43 2022] GET /identity/v3/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 223 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:43.464048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:43.505564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:43.566820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836bae4c-9a1e-4238-9e00-b6232c532c73 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:43.602199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd09978-2432-4346-abe4-070002309330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:43.602832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd09978-2432-4346-abe4-070002309330 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:43.603113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd09978-2432-4346-abe4-070002309330 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:43.603389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd09978-2432-4346-abe4-070002309330 None admin] PATH_INFO: `/v3/users/c3152430394e49a892c44409af03d083` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:43.616866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd09978-2432-4346-abe4-070002309330 None admin] RBAC: Authorizing `identity:get_user(user_id=c3152430394e49a892c44409af03d083)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:43.618360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd09978-2432-4346-abe4-070002309330 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:43.620032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 560/1114] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:43 2022] GET /identity/v3/users/c3152430394e49a892c44409af03d083 => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:43.626370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:43.727132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9831c7c-68f9-47b1-ad01-88d192cb6346 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:43.747101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:43.761189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:43.761790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:43.762064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:43.762334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:43.831974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:43.833396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:43.938540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 555/1115] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:43.944621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd09978-2432-4346-abe4-070002309330 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:44.042149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:44.049494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd09978-2432-4346-abe4-070002309330 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:44.087150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0b7d74-1b36-4866-bfa0-bd9b4791c72c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:44.087150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7d74-1b36-4866-bfa0-bd9b4791c72c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:44.087150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7d74-1b36-4866-bfa0-bd9b4791c72c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:44.087150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0b7d74-1b36-4866-bfa0-bd9b4791c72c None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:44.088261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b7d74-1b36-4866-bfa0-bd9b4791c72c None admin] RBAC: Authorizing `identity:get_project(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:44.089687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b0b7d74-1b36-4866-bfa0-bd9b4791c72c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:44.091399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 561/1116] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:43 2022] GET /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:44.097945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:44.119893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:44.196783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1692de6-88bf-46e5-94ae-e5b989a2ba2b None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:44.231428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:44.232555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:44.232837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:44.233139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:44.233898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] RBAC: Adding query filter params (role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe, user.id=c3152430394e49a892c44409af03d083) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:44.234198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:44.235600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d50622-9a31-4501-8e65-482c5e5b67fd None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:44.240890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 556/1117] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:44 2022] GET /identity/v3/role_assignments?user.id=c3152430394e49a892c44409af03d083&scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:46.230367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69337-ea1d-4b5d-b275-70098eaffff6 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:46.230928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69337-ea1d-4b5d-b275-70098eaffff6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:46.230928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b69337-ea1d-4b5d-b275-70098eaffff6 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:46.232807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 562/1118] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:46.238385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:46.238813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:46.238911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:46.257972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:46.259240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:46.340183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 557/1119] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:46.612014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b69337-ea1d-4b5d-b275-70098eaffff6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:46.712716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b69337-ea1d-4b5d-b275-70098eaffff6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:46.747094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:46.747875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:46.748051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:46.748323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] PATH_INFO: `/v3/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:46.749742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] RBAC: Authorizing `identity:get_role(role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:46.751110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:46.752440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 563/1120] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:46 2022] GET /identity/v3/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:46.758440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:46.773620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:46.858243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb0a042-f940-4799-b523-0ccf29c7d917 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:46.891803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:46.892430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:46.892712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:46.892985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] PATH_INFO: `/v3/users/c3152430394e49a892c44409af03d083` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:46.894147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] RBAC: Authorizing `identity:get_user(user_id=c3152430394e49a892c44409af03d083)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:46.895563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:46.897154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 558/1121] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:46 2022] GET /identity/v3/users/c3152430394e49a892c44409af03d083 => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:46.903049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:46.936863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:47.013781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94004a-ede2-4d98-b3fc-db506bfefec6 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:47.052083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6874f0f-375e-49d3-81ae-02a2351dc84d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:47.052736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6874f0f-375e-49d3-81ae-02a2351dc84d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:47.053049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6874f0f-375e-49d3-81ae-02a2351dc84d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:47.053323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6874f0f-375e-49d3-81ae-02a2351dc84d None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:47.054506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6874f0f-375e-49d3-81ae-02a2351dc84d None admin] RBAC: Authorizing `identity:get_project(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:47.056013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6874f0f-375e-49d3-81ae-02a2351dc84d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:47.057811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 564/1122] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:46 2022] GET /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:47.064321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:47.151658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:47.171376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6328d58f-4702-44ed-9a36-37c014125b17 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:47.207746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184db49c-6c73-4678-b34f-e952875cba02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:47.208416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184db49c-6c73-4678-b34f-e952875cba02 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:47.208695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184db49c-6c73-4678-b34f-e952875cba02 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:47.208970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184db49c-6c73-4678-b34f-e952875cba02 None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe/users/c3152430394e49a892c44409af03d083/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:47.211407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184db49c-6c73-4678-b34f-e952875cba02 None admin] RBAC: Authorizing `identity:create_grant(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe, user_id=c3152430394e49a892c44409af03d083, role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:47.212845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184db49c-6c73-4678-b34f-e952875cba02 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:47.292190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 559/1123] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:11:47 2022] PUT /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe/users/c3152430394e49a892c44409af03d083/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:11:49.213476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d4c89-2c86-4a20-8856-b25040cc2a05 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:49.214069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d4c89-2c86-4a20-8856-b25040cc2a05 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:49.214069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56d4c89-2c86-4a20-8856-b25040cc2a05 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:49.215368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 565/1124] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:49.220944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:49.221424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:49.221517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:49.240638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:49.241884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:49.309639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 560/1125] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:49.583732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d4c89-2c86-4a20-8856-b25040cc2a05 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:49.689085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:49.693224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56d4c89-2c86-4a20-8856-b25040cc2a05 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:49.732921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:49.733590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:49.733867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:49.734136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] PATH_INFO: `/v3/roles/cbba7c3b564a4c9d9cb589456f933e52` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:49.735544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] RBAC: Authorizing `identity:get_role(role_id=cbba7c3b564a4c9d9cb589456f933e52)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:49.737046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:49.738393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 566/1126] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:49 2022] GET /identity/v3/roles/cbba7c3b564a4c9d9cb589456f933e52 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:49.744367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:49.815729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:49.844679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3b31e7-655c-415a-bf0c-3de46a6613f6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:49.879789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:49.892028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:49.892028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:49.892028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] PATH_INFO: `/v3/users/c3152430394e49a892c44409af03d083` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:49.892028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] RBAC: Authorizing `identity:get_user(user_id=c3152430394e49a892c44409af03d083)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:49.892028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:49.892028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 561/1127] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:49 2022] GET /identity/v3/users/c3152430394e49a892c44409af03d083 => generated 286 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:49.892549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:49.991946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b858792-7026-4d04-ae16-4b7ebf8a17b4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:50.026912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:50.027514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:50.027794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:50.028068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:50.067739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:50.104081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:50.105588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:50.210884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 567/1128] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:49 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:50.217260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:50.324491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c1b033-23ee-4656-b5b1-59e734f06962 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:50.361314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6143ce-8622-4108-9cc0-80c6ed4754fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:50.361977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6143ce-8622-4108-9cc0-80c6ed4754fa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:50.362251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6143ce-8622-4108-9cc0-80c6ed4754fa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:50.362520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6143ce-8622-4108-9cc0-80c6ed4754fa None admin] PATH_INFO: `/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:50.363752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6143ce-8622-4108-9cc0-80c6ed4754fa None admin] RBAC: Authorizing `identity:get_project(project_id=097c69b6ff4c45c1b9150fa4a8b8dbfe)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:50.365240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f6143ce-8622-4108-9cc0-80c6ed4754fa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:50.367104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 562/1129] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:50 2022] GET /identity/v3/projects/097c69b6ff4c45c1b9150fa4a8b8dbfe => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:50.374310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:50.442843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:50.476113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d2ac1b-3ffb-4777-be32-a95c1cafd219 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:50.511310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:50.511917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:50.512195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:50.512467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:50.513805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] RBAC: Adding query filter params (role.id=cbba7c3b564a4c9d9cb589456f933e52, scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe, user.id=c3152430394e49a892c44409af03d083) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:50.514380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:50.515966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ae012-1239-46a5-b223-3108d5fe1b7f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:50.521896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 568/1130] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:50 2022] GET /identity/v3/role_assignments?user.id=c3152430394e49a892c44409af03d083&scope.project.id=097c69b6ff4c45c1b9150fa4a8b8dbfe&role.id=cbba7c3b564a4c9d9cb589456f933e52 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:52.499720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb06d02-e769-44f8-9e2f-2025ec400205 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:52.500430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb06d02-e769-44f8-9e2f-2025ec400205 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:52.500430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb06d02-e769-44f8-9e2f-2025ec400205 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:52.501639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 563/1131] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:52.507170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:52.507632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:52.507765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:52.526478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:52.527659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:52.598816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 569/1132] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:52 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:52.875631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb06d02-e769-44f8-9e2f-2025ec400205 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:52.886670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:52.983235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb06d02-e769-44f8-9e2f-2025ec400205 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:53.017662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e85344-da86-4b56-b162-2ce5fbe91a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:53.018297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e85344-da86-4b56-b162-2ce5fbe91a8e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:53.018578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e85344-da86-4b56-b162-2ce5fbe91a8e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:53.018849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e85344-da86-4b56-b162-2ce5fbe91a8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:53.019992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e85344-da86-4b56-b162-2ce5fbe91a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:53.021452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e85344-da86-4b56-b162-2ce5fbe91a8e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:53.023286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 564/1133] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:53.029722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:53.094201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:53.145644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045c35e1-2fdc-4363-8e9d-3e3f19e743a6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:53.182144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459a6507-b8dd-4246-bccf-22dbe0413801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:53.182753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6507-b8dd-4246-bccf-22dbe0413801 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:53.183031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6507-b8dd-4246-bccf-22dbe0413801 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:53.183301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459a6507-b8dd-4246-bccf-22dbe0413801 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:53.183944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a6507-b8dd-4246-bccf-22dbe0413801 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:53.185393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459a6507-b8dd-4246-bccf-22dbe0413801 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:53.271427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 570/1134] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:11:53 2022] POST /identity/v3/projects => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:55.241964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871112a8-77d1-45ed-a799-58abd5cf0f54 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:55.242524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871112a8-77d1-45ed-a799-58abd5cf0f54 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:55.242625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871112a8-77d1-45ed-a799-58abd5cf0f54 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:55.244009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 565/1135] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:55.250901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:55.251406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:55.251496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:55.269681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:55.270882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:55.339060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 571/1136] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:55 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:55.617871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871112a8-77d1-45ed-a799-58abd5cf0f54 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:55.701628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:55.726182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871112a8-77d1-45ed-a799-58abd5cf0f54 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:55.761206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fb3f2d-7571-48a6-a38f-199fbde43389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:55.761830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3f2d-7571-48a6-a38f-199fbde43389 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:55.762135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3f2d-7571-48a6-a38f-199fbde43389 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:55.762408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fb3f2d-7571-48a6-a38f-199fbde43389 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:55.763630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb3f2d-7571-48a6-a38f-199fbde43389 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:55.765100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fb3f2d-7571-48a6-a38f-199fbde43389 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:55.766660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 566/1137] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:11:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:55.772553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:55.820088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:55.874926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829f4fa-dac8-4212-b364-acb3bd96fb8e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:55.909172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a45e90-fcff-423b-959f-09138db95fb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:55.909782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45e90-fcff-423b-959f-09138db95fb6 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:55.910066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45e90-fcff-423b-959f-09138db95fb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:55.910337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a45e90-fcff-423b-959f-09138db95fb6 None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:55.910999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45e90-fcff-423b-959f-09138db95fb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:55.912424 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a45e90-fcff-423b-959f-09138db95fb6 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:56.018728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 572/1138] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:11:55 2022] POST /identity/v3/users => generated 286 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:11:57.997020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c97c1-af54-4266-8d21-bc38cc63c4aa None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:57.997555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c97c1-af54-4266-8d21-bc38cc63c4aa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:57.997665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2c97c1-af54-4266-8d21-bc38cc63c4aa None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:57.998987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 567/1139] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:11:58.004938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:58.005435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:58.005536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:58.023108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:11:58.024316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:11:58.091411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 573/1140] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:11:58 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:11:58.379676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c97c1-af54-4266-8d21-bc38cc63c4aa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:58.483467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2c97c1-af54-4266-8d21-bc38cc63c4aa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:58.518610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:58.519226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:58.519500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:58.519766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:58.529882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:11:58.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:11:58.536777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:11:58.538898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 568/1141] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:11:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:11:58.543866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:58.575109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:58.644347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93b50e-3ff7-4888-98b2-2369f74e6e6f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:58.679509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:58.680115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:58.680394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:58.680664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:58.681353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:58.681656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:58.683018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:58.691552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 574/1142] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:11:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:11:58.697517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:58.701974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:58.799879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2356446-da9c-4672-b687-7fa794f9bad9 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:58.833507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:58.834143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:58.834422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:58.834696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] PATH_INFO: `/v3/users/86c11033cd8c452f8f1d7be29fbc3d79` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:58.849328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] RBAC: Authorizing `identity:get_user(user_id=86c11033cd8c452f8f1d7be29fbc3d79)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:58.850773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:58.852439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 569/1143] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:11:58 2022] GET /identity/v3/users/86c11033cd8c452f8f1d7be29fbc3d79 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:58.859631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:58.950051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:58.958593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee6a312-73d9-4226-aaed-6d66bf71b7d1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:58.992985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:58.993607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:58.993887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:58.994159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.060544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:59.062686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:59.151310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:59.160879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 575/1144] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:11:58 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:11:59.167029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:59.229313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:11:59.267673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c36c24-3cb3-4a4a-93f2-49660b8028d0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:59.303036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b5de76-8741-43a0-801c-aecfc78ed341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:59.303660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5de76-8741-43a0-801c-aecfc78ed341 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:59.303939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5de76-8741-43a0-801c-aecfc78ed341 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:59.304210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b5de76-8741-43a0-801c-aecfc78ed341 None admin] PATH_INFO: `/v3/projects/5366ebb1aae14b11bdfbc261e2ced723` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.305347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5de76-8741-43a0-801c-aecfc78ed341 None admin] RBAC: Authorizing `identity:get_project(project_id=5366ebb1aae14b11bdfbc261e2ced723)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:59.306777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b5de76-8741-43a0-801c-aecfc78ed341 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:59.308446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 570/1145] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:11:59 2022] GET /identity/v3/projects/5366ebb1aae14b11bdfbc261e2ced723 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:11:59.315502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:11:59.426026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc7f88-3c4f-4286-8c9a-fb5a34785621 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:11:59.462450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:11:59.463071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:11:59.463345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:11:59.463614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:11:59.464362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=5366ebb1aae14b11bdfbc261e2ced723, user.id=86c11033cd8c452f8f1d7be29fbc3d79) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:11:59.464670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:11:59.466175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f4a4f5-913b-449c-bed8-3c3a603d0a61 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:11:59.473344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 576/1146] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:11:59 2022] GET /identity/v3/role_assignments?user.id=86c11033cd8c452f8f1d7be29fbc3d79&scope.project.id=5366ebb1aae14b11bdfbc261e2ced723&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:01.435000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a84ca-43b3-4acc-a4bd-47116a7b7087 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:01.435531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a84ca-43b3-4acc-a4bd-47116a7b7087 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:01.435623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007a84ca-43b3-4acc-a4bd-47116a7b7087 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:01.437498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 571/1147] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:01.442746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:01.443181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:01.443291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:01.461323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:01.462505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:01.518466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:01.535141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 577/1148] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:01 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:01.809074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a84ca-43b3-4acc-a4bd-47116a7b7087 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:01.909519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007a84ca-43b3-4acc-a4bd-47116a7b7087 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:01.938943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:01.944501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:01.945157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:01.945437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:01.945708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:01.955713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:01.960392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:01.967924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:12:01.968863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 572/1149] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:01.969613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:02.068982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafaa5-8c9b-472d-bb3f-62eea54b9d05 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:02.103799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:02.104415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:02.104696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:02.104971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:02.105675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:02.105981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:02.107409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:02.115759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 578/1150] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:02.121677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:02.224447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a8e0b-57e4-4e56-8ebe-129bea967656 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:02.258488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:02.264116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:02.264116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:02.264116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] PATH_INFO: `/v3/users/86c11033cd8c452f8f1d7be29fbc3d79` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:02.264116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] RBAC: Authorizing `identity:get_user(user_id=86c11033cd8c452f8f1d7be29fbc3d79)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:02.264116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:02.264677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 573/1151] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:02 2022] GET /identity/v3/users/86c11033cd8c452f8f1d7be29fbc3d79 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:02.270613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:02.319826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:02.372179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-298e013c-7119-45f4-8d5e-8150fb0f516d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:02.406655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d47a521-9405-475a-8268-f63c320bb465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:02.407279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47a521-9405-475a-8268-f63c320bb465 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:02.407555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47a521-9405-475a-8268-f63c320bb465 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:02.407821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d47a521-9405-475a-8268-f63c320bb465 None admin] PATH_INFO: `/v3/projects/5366ebb1aae14b11bdfbc261e2ced723` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:02.409038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47a521-9405-475a-8268-f63c320bb465 None admin] RBAC: Authorizing `identity:get_project(project_id=5366ebb1aae14b11bdfbc261e2ced723)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:02.410569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d47a521-9405-475a-8268-f63c320bb465 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:02.412188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 579/1152] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:02 2022] GET /identity/v3/projects/5366ebb1aae14b11bdfbc261e2ced723 => generated 311 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:02.418260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:02.478688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:02.524458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20606d22-8143-4632-a5fe-289f3402f8fc None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:02.563085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4086fc-e8eb-46dc-9407-4053fd0f3c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:02.563758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4086fc-e8eb-46dc-9407-4053fd0f3c8d None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:02.564041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4086fc-e8eb-46dc-9407-4053fd0f3c8d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:02.564314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4086fc-e8eb-46dc-9407-4053fd0f3c8d None admin] PATH_INFO: `/v3/projects/5366ebb1aae14b11bdfbc261e2ced723/users/86c11033cd8c452f8f1d7be29fbc3d79/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:02.566293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4086fc-e8eb-46dc-9407-4053fd0f3c8d None admin] RBAC: Authorizing `identity:create_grant(project_id=5366ebb1aae14b11bdfbc261e2ced723, user_id=86c11033cd8c452f8f1d7be29fbc3d79, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:02.567719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4086fc-e8eb-46dc-9407-4053fd0f3c8d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:02.646779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 574/1153] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:12:02 2022] PUT /identity/v3/projects/5366ebb1aae14b11bdfbc261e2ced723/users/86c11033cd8c452f8f1d7be29fbc3d79/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:12:04.551578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff4822-9737-49e5-ad54-3f9b0f0cb6b3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:04.552106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff4822-9737-49e5-ad54-3f9b0f0cb6b3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:04.552106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ff4822-9737-49e5-ad54-3f9b0f0cb6b3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:04.553512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 580/1154] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:04.559883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:04.562203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:04.562203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:04.586817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:04.588007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:04.658156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 575/1155] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:04.926035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff4822-9737-49e5-ad54-3f9b0f0cb6b3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.025975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ff4822-9737-49e5-ad54-3f9b0f0cb6b3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.042864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:05.059743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.060450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.063233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.063233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.070861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:05.076458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:05.077603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:12:05.078826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 581/1156] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:05.083677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.132161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:05.182928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe1bc31-5b31-4a04-9fe5-5f165855b062 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.217953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.218554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.218833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.219104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.219772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:05.220074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:05.221468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:05.229299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 576/1157] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:05.235466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.333709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9110fa-b97f-420c-8dec-cbbd65b8d0d1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.366733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.367352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.367630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.367902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] PATH_INFO: `/v3/users/86c11033cd8c452f8f1d7be29fbc3d79` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.369005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] RBAC: Authorizing `identity:get_user(user_id=86c11033cd8c452f8f1d7be29fbc3d79)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:05.370447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:05.372010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 582/1158] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:05 2022] GET /identity/v3/users/86c11033cd8c452f8f1d7be29fbc3d79 => generated 286 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:05.378569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.477196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e0de7-29e7-4b35-8b62-18cef0302947 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.487582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:05.510853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.511459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.511740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.512039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.580461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:05.581980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:05.679746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 577/1159] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:12:05 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:12:05.686380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.733304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:05.787092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec863e5-1127-452e-995a-51d7d8ee9e8e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.820599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82c1756-88f8-4379-92e0-5ae2ae0ae3b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.821248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1756-88f8-4379-92e0-5ae2ae0ae3b4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.821526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1756-88f8-4379-92e0-5ae2ae0ae3b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.821797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82c1756-88f8-4379-92e0-5ae2ae0ae3b4 None admin] PATH_INFO: `/v3/projects/5366ebb1aae14b11bdfbc261e2ced723` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.822904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c1756-88f8-4379-92e0-5ae2ae0ae3b4 None admin] RBAC: Authorizing `identity:get_project(project_id=5366ebb1aae14b11bdfbc261e2ced723)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:05.824331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82c1756-88f8-4379-92e0-5ae2ae0ae3b4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:05.826024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 583/1160] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:05 2022] GET /identity/v3/projects/5366ebb1aae14b11bdfbc261e2ced723 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:05.832196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:05.843601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:05.931730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3855f99e-00f1-468a-8c98-6e5f24025869 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:05.965525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:05.966123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:05.966437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:05.967304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:05.968033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=5366ebb1aae14b11bdfbc261e2ced723, user.id=86c11033cd8c452f8f1d7be29fbc3d79) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:05.968339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:05.969794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8824fbf-fee2-4def-8fe0-5336140ce0a3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:05.975423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 578/1161] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:12:05 2022] GET /identity/v3/role_assignments?user.id=86c11033cd8c452f8f1d7be29fbc3d79&scope.project.id=5366ebb1aae14b11bdfbc261e2ced723&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:07.931038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0163a8-7e3c-4c2b-bc85-1e52e859d77c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:07.931617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0163a8-7e3c-4c2b-bc85-1e52e859d77c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:07.931617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b0163a8-7e3c-4c2b-bc85-1e52e859d77c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:07.932875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 584/1162] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:12:07.939836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:07.940387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:07.940387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:07.959397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:07.960611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:08.033979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 579/1163] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:07 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:08.301661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0163a8-7e3c-4c2b-bc85-1e52e859d77c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.412399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b0163a8-7e3c-4c2b-bc85-1e52e859d77c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.426786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:08.447628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.448247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.448527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.448802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.527166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:08.528627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:08.638279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 585/1164] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:12:08 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:12:08.645049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.648535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:08.759795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799c3a6-2208-4923-b5e7-a9ed63482a40 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.798895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.799545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.799825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.800094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.810519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:08.813256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-e8daa29d-6f24-40b0-8516-fc9aa69f4d55 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:08.836456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:08.838419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:12:08.839736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 580/1165] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:12:08 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:08.845331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:08.861914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:08.956994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6726be-cb48-4dc5-95e5-5ab7f58fae10 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:08.994653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:08.995263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:08.995543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:08.995821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:08.996471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:08.996779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:08.998869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7931a6b0-70c7-46ce-8bba-c33302f87889 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:09.006714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 586/1166] 10.222.0.73 () {56 vars in 1115 bytes} [Tue Aug 23 22:12:08 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:10.967926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea115e3-cd85-4d27-98b9-860bae79ee81 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:10.968490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea115e3-cd85-4d27-98b9-860bae79ee81 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:10.968670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea115e3-cd85-4d27-98b9-860bae79ee81 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:10.970206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 581/1167] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:10.976184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2003c90-31e5-40b6-bdef-1b0b1b4f2323 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:10.976636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2003c90-31e5-40b6-bdef-1b0b1b4f2323 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:10.976724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2003c90-31e5-40b6-bdef-1b0b1b4f2323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:10.995759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e2003c90-31e5-40b6-bdef-1b0b1b4f2323 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:10.996958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2003c90-31e5-40b6-bdef-1b0b1b4f2323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:11.074920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 587/1168] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:10 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:11.354984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea115e3-cd85-4d27-98b9-860bae79ee81 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:11.421770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:11.460565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea115e3-cd85-4d27-98b9-860bae79ee81 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:11.498116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd2ba56-7f19-46a0-b46d-04e79f6b2474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:11.498901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ba56-7f19-46a0-b46d-04e79f6b2474 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:11.499185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ba56-7f19-46a0-b46d-04e79f6b2474 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:11.499459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd2ba56-7f19-46a0-b46d-04e79f6b2474 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:11.500221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2ba56-7f19-46a0-b46d-04e79f6b2474 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:11.501647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd2ba56-7f19-46a0-b46d-04e79f6b2474 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:11.584261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 582/1169] 10.222.0.73 () {60 vars in 1138 bytes} [Tue Aug 23 22:12:11 2022] POST /identity/v3/domains => generated 267 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:12:13.584087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe85e07-4ec6-4a7f-ac24-7102004b6723 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:13.584589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe85e07-4ec6-4a7f-ac24-7102004b6723 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:13.584715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe85e07-4ec6-4a7f-ac24-7102004b6723 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:13.586032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 588/1170] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:13.591786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:13.592388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:13.592388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:13.611522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:13.612698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:13.682888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 583/1171] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:13.950756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe85e07-4ec6-4a7f-ac24-7102004b6723 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:13.959606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:14.056815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe85e07-4ec6-4a7f-ac24-7102004b6723 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:14.096276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b78d18a-44b6-4dd4-b919-b54ec954d5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:14.097124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78d18a-44b6-4dd4-b919-b54ec954d5b9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:14.097401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78d18a-44b6-4dd4-b919-b54ec954d5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:14.097678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b78d18a-44b6-4dd4-b919-b54ec954d5b9 None admin] PATH_INFO: `/v3/domains/1f06c44c25924805b0bbb219cf0d2741` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:14.110414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78d18a-44b6-4dd4-b919-b54ec954d5b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f06c44c25924805b0bbb219cf0d2741)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:14.112324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b78d18a-44b6-4dd4-b919-b54ec954d5b9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:14.114774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 589/1172] 10.222.0.73 () {56 vars in 1183 bytes} [Tue Aug 23 22:12:13 2022] GET /identity/v3/domains/1f06c44c25924805b0bbb219cf0d2741 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:14.120611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:14.175731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:14.217665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c82eebe-b30c-4196-b32e-814c0409c338 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:14.251382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb2592-62c8-428e-b8e5-b36b62025338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:14.251979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2592-62c8-428e-b8e5-b36b62025338 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:14.252270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2592-62c8-428e-b8e5-b36b62025338 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:14.252517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb2592-62c8-428e-b8e5-b36b62025338 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:14.253177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb2592-62c8-428e-b8e5-b36b62025338 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:14.254588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb2592-62c8-428e-b8e5-b36b62025338 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:14.355781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 584/1173] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:12:14 2022] POST /identity/v3/projects => generated 361 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:12:16.326758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38835f2-6fb3-4ecb-b18c-827f466ec781 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:16.327353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38835f2-6fb3-4ecb-b18c-827f466ec781 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:16.327353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38835f2-6fb3-4ecb-b18c-827f466ec781 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:16.328645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 590/1174] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:16.333930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:16.334633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:16.334857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:16.354040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:16.355238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:16.424064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 585/1175] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:16 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:16.702814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38835f2-6fb3-4ecb-b18c-827f466ec781 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:16.773889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:16.810687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38835f2-6fb3-4ecb-b18c-827f466ec781 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:16.847479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd31fa5-77e9-494c-bd9c-28f975315922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:16.848119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31fa5-77e9-494c-bd9c-28f975315922 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:16.848402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31fa5-77e9-494c-bd9c-28f975315922 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:16.848672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd31fa5-77e9-494c-bd9c-28f975315922 None admin] PATH_INFO: `/v3/domains/1f06c44c25924805b0bbb219cf0d2741` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:16.849840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31fa5-77e9-494c-bd9c-28f975315922 None admin] RBAC: Authorizing `identity:get_domain(domain_id=1f06c44c25924805b0bbb219cf0d2741)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:16.851260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd31fa5-77e9-494c-bd9c-28f975315922 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:16.852848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 591/1176] 10.222.0.73 () {56 vars in 1183 bytes} [Tue Aug 23 22:12:16 2022] GET /identity/v3/domains/1f06c44c25924805b0bbb219cf0d2741 => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:16.859821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:16.893866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:16.960478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bb74e7-a063-432c-adfe-a4ac7267f9e2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:16.996606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922dea1-6aa5-43ad-a4f6-c26e1990bad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:16.997322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922dea1-6aa5-43ad-a4f6-c26e1990bad2 None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:16.997606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922dea1-6aa5-43ad-a4f6-c26e1990bad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:16.997881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922dea1-6aa5-43ad-a4f6-c26e1990bad2 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:16.998526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922dea1-6aa5-43ad-a4f6-c26e1990bad2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:16.999952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b922dea1-6aa5-43ad-a4f6-c26e1990bad2 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:17.126359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 586/1177] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:12:16 2022] POST /identity/v3/users => generated 311 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:12:19.108180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8916-9d68-4083-b7e4-f7753f4a94fd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:19.108741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8916-9d68-4083-b7e4-f7753f4a94fd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:19.108741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900a8916-9d68-4083-b7e4-f7753f4a94fd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:19.110081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 592/1178] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:19.115416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:19.115808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:19.116040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:19.135530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:19.136723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:19.206995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 587/1179] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:19.491131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8916-9d68-4083-b7e4-f7753f4a94fd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:19.599956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900a8916-9d68-4083-b7e4-f7753f4a94fd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:19.633775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:19.634396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:19.634671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:19.634939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:19.636746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:19.645542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:19.656672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:19.658145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:12:19.659292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 593/1180] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:19.660241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:19.682541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:19.758958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e843d6-a0c4-4912-ac3f-fcefce827da3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:19.792653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:19.793285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:19.793562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:19.793843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:19.795417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:19.795719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:19.797121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:19.805500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 588/1181] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:19.811644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:19.909189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e726fe69-7b5b-44e1-9291-137ddeddb1a8 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:19.942341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:19.942960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:19.943236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:19.943503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] PATH_INFO: `/v3/users/bbb18a9590b5497bac96ed507f8fbffb` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:19.956764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] RBAC: Authorizing `identity:get_user(user_id=bbb18a9590b5497bac96ed507f8fbffb)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:19.958211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:19.959790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 594/1182] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:19 2022] GET /identity/v3/users/bbb18a9590b5497bac96ed507f8fbffb => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:19.965859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:20.050825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:20.066096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851ef75b-ea9f-44a0-8d5f-1b461431ea0e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:20.099259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:20.099847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:20.100122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:20.100386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:20.164500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:20.165952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:20.244184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:20.259451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 589/1183] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:12:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:12:20.266257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:20.318862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:20.365413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ac43e8-cdaa-4a3f-889a-8a0c753f134a None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:20.399159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a85ab-9c88-4c39-9a6e-2f1bd636b3c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:20.399784 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a85ab-9c88-4c39-9a6e-2f1bd636b3c4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:20.400063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a85ab-9c88-4c39-9a6e-2f1bd636b3c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:20.400332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a85ab-9c88-4c39-9a6e-2f1bd636b3c4 None admin] PATH_INFO: `/v3/projects/9404bfd842df498bbf3e580efbc563a2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:20.401464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a85ab-9c88-4c39-9a6e-2f1bd636b3c4 None admin] RBAC: Authorizing `identity:get_project(project_id=9404bfd842df498bbf3e580efbc563a2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:20.402884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a85ab-9c88-4c39-9a6e-2f1bd636b3c4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:20.404603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 595/1184] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:20 2022] GET /identity/v3/projects/9404bfd842df498bbf3e580efbc563a2 => generated 361 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:20.411346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:20.510748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc49128-6e06-46eb-b6b9-286661d85d83 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:20.544832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:20.545699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:20.546124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:20.546402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:20.547157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=9404bfd842df498bbf3e580efbc563a2, user.id=bbb18a9590b5497bac96ed507f8fbffb) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:20.547467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:20.548895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1535415b-e483-47f3-a64e-96d6dd6c978c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:20.554810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 590/1185] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:12:20 2022] GET /identity/v3/role_assignments?user.id=bbb18a9590b5497bac96ed507f8fbffb&scope.project.id=9404bfd842df498bbf3e580efbc563a2&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:22.433648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a03d-bc10-47bd-a5e9-b5ae13f3577c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:22.434179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a03d-bc10-47bd-a5e9-b5ae13f3577c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:22.434179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7984a03d-bc10-47bd-a5e9-b5ae13f3577c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:22.436214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 596/1186] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:22.441435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07150301-f550-459b-9450-88af50d30d70 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:22.441861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07150301-f550-459b-9450-88af50d30d70 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:22.441965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07150301-f550-459b-9450-88af50d30d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:22.459979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-07150301-f550-459b-9450-88af50d30d70 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:22.461214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-07150301-f550-459b-9450-88af50d30d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:22.506494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:22.530440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 591/1187] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:22 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:22.803025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984a03d-bc10-47bd-a5e9-b5ae13f3577c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:22.903561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7984a03d-bc10-47bd-a5e9-b5ae13f3577c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:22.924331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:22.937653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:22.938278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:22.940013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:22.940013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:22.948526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:22.953914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:22.955018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:12:22.956219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 597/1188] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:22.961376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07150301-f550-459b-9450-88af50d30d70 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:23.058239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07150301-f550-459b-9450-88af50d30d70 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:23.092577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:23.093222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:23.093508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:23.093779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:23.094451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:23.094754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:23.096112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:23.104331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 592/1189] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:23.110398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:23.207157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a9be61-2fc7-47ec-9b6a-a92dc4aeed06 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:23.241317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:23.241941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:23.242221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:23.242493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] PATH_INFO: `/v3/users/bbb18a9590b5497bac96ed507f8fbffb` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:23.243614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] RBAC: Authorizing `identity:get_user(user_id=bbb18a9590b5497bac96ed507f8fbffb)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:23.245057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:23.246636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 598/1190] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:23 2022] GET /identity/v3/users/bbb18a9590b5497bac96ed507f8fbffb => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:23.252578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:23.296130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:23.353004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65601e19-e58b-4cfe-8b09-224d9a9fec5a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:23.385820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0544acc-a1cb-4575-a211-8dc91324ef5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:23.386461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544acc-a1cb-4575-a211-8dc91324ef5c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:23.386719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544acc-a1cb-4575-a211-8dc91324ef5c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:23.387102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0544acc-a1cb-4575-a211-8dc91324ef5c None admin] PATH_INFO: `/v3/projects/9404bfd842df498bbf3e580efbc563a2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:23.388227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544acc-a1cb-4575-a211-8dc91324ef5c None admin] RBAC: Authorizing `identity:get_project(project_id=9404bfd842df498bbf3e580efbc563a2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:23.390420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0544acc-a1cb-4575-a211-8dc91324ef5c None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:23.392050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 593/1191] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:23 2022] GET /identity/v3/projects/9404bfd842df498bbf3e580efbc563a2 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:23.398153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:23.447549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:23.497417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd827e9-dced-417c-a451-8ff280cf2631 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:23.531702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ace140-7483-4739-8830-d9d978760e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:23.532350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ace140-7483-4739-8830-d9d978760e79 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:23.532627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ace140-7483-4739-8830-d9d978760e79 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:23.532899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ace140-7483-4739-8830-d9d978760e79 None admin] PATH_INFO: `/v3/projects/9404bfd842df498bbf3e580efbc563a2/users/bbb18a9590b5497bac96ed507f8fbffb/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:23.534806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ace140-7483-4739-8830-d9d978760e79 None admin] RBAC: Authorizing `identity:create_grant(project_id=9404bfd842df498bbf3e580efbc563a2, user_id=bbb18a9590b5497bac96ed507f8fbffb, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:23.536225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ace140-7483-4739-8830-d9d978760e79 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:23.616738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 599/1192] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:12:23 2022] PUT /identity/v3/projects/9404bfd842df498bbf3e580efbc563a2/users/bbb18a9590b5497bac96ed507f8fbffb/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:12:25.505891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4bcff-562f-44fa-bbfb-26a130f40ab9 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:25.505891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4bcff-562f-44fa-bbfb-26a130f40ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:25.506536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c4bcff-562f-44fa-bbfb-26a130f40ab9 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:25.507560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 594/1193] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:12:25.515377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:25.515900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:25.516287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:25.534899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:12:25.536122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:12:25.603589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 600/1194] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:12:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:12:25.872111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4bcff-562f-44fa-bbfb-26a130f40ab9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:25.973981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c4bcff-562f-44fa-bbfb-26a130f40ab9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:25.985645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:26.010184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:26.010815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:26.011087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:26.011352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:26.021937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:12:26.026686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:12:26.027646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:12:26.028485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:12:26.029394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 595/1195] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:12:26.034432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:26.074544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:26.137449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90b1cb6-1b41-4234-8117-da06bda08c71 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:26.171374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:26.171977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:26.172259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:26.172533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:26.173223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:26.173532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:26.174904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:26.182741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 601/1196] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:12:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:26.188925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:26.288373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761c82d8-b967-417f-ac9c-f3666205afd7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:26.320682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:26.321335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:26.321613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:26.321882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] PATH_INFO: `/v3/users/bbb18a9590b5497bac96ed507f8fbffb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:26.323020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] RBAC: Authorizing `identity:get_user(user_id=bbb18a9590b5497bac96ed507f8fbffb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:26.324430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:26.326522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 596/1197] 10.222.0.73 () {56 vars in 1177 bytes} [Tue Aug 23 22:12:26 2022] GET /identity/v3/users/bbb18a9590b5497bac96ed507f8fbffb => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:26.332573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:26.428351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:26.431398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3918dc30-f34c-427b-95c5-bdfa57441c36 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:26.464864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:26.465498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:26.465779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:26.466076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:26.534966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:26.536801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:26.634311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:26.637908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 602/1198] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:12:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:12:26.644577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:26.679729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:26.743626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac97727-f399-4ad9-8c84-c86fbf558828 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:26.778009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3ddd4a-fc25-4ea0-aa4d-3a70d5c1fd5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:26.778647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddd4a-fc25-4ea0-aa4d-3a70d5c1fd5d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:26.778922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddd4a-fc25-4ea0-aa4d-3a70d5c1fd5d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:26.779246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3ddd4a-fc25-4ea0-aa4d-3a70d5c1fd5d None admin] PATH_INFO: `/v3/projects/9404bfd842df498bbf3e580efbc563a2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:26.780833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ddd4a-fc25-4ea0-aa4d-3a70d5c1fd5d None admin] RBAC: Authorizing `identity:get_project(project_id=9404bfd842df498bbf3e580efbc563a2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:26.782272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3ddd4a-fc25-4ea0-aa4d-3a70d5c1fd5d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:26.783889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 597/1199] 10.222.0.73 () {56 vars in 1186 bytes} [Tue Aug 23 22:12:26 2022] GET /identity/v3/projects/9404bfd842df498bbf3e580efbc563a2 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:26.790075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:12:26.887687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8a61a1-b605-47f3-9d33-87b196eb81ae None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:12:26.922016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:12:26.922637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:12:26.922921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:12:26.923193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:12:26.923911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=9404bfd842df498bbf3e580efbc563a2, user.id=bbb18a9590b5497bac96ed507f8fbffb) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:12:26.924220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:12:26.925686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20418b6b-3980-4a53-8a87-24fd1f40838c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:12:26.931435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 603/1200] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:12:26 2022] GET /identity/v3/role_assignments?user.id=bbb18a9590b5497bac96ed507f8fbffb&scope.project.id=9404bfd842df498bbf3e580efbc563a2&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:12:36.772260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=48 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:12:36.928868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=68 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:48.768763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20539f9e-e9f4-4e20-907a-2bf5107f9513 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:48.769393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20539f9e-e9f4-4e20-907a-2bf5107f9513 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:48.769393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20539f9e-e9f4-4e20-907a-2bf5107f9513 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:48.770740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 598/1201] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:48.776191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:48.776632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:48.776735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:48.795742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:48.796913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:48.863072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 604/1202] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:49.130838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20539f9e-e9f4-4e20-907a-2bf5107f9513 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:49.236075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20539f9e-e9f4-4e20-907a-2bf5107f9513 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:49.270719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:49.271375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:49.271652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:49.271918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:49.282852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:49.284271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:49.297698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:49.298812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:49.299960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:49.299960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:49.299960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:13:49.299960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 599/1203] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:49.304926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:49.404308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dfca08-5461-44bb-b249-0d9528907866 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:49.432162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:49.438168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:49.438772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:49.439052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:49.439345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:49.439967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:49.440271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:49.441670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320b75ba-dfa3-41ee-8ac8-dc47da04d662 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:49.452508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 605/1204] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:49.458487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:49.513817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:49.560813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543b8d5-af89-41bc-bb89-593ad6dcde68 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:49.595127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a5a882-d58e-453d-b4a0-15d5d7e0e51a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:49.595734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5a882-d58e-453d-b4a0-15d5d7e0e51a None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:49.596014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5a882-d58e-453d-b4a0-15d5d7e0e51a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:49.596285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a5a882-d58e-453d-b4a0-15d5d7e0e51a None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:49.596928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5a882-d58e-453d-b4a0-15d5d7e0e51a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:49.598383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a5a882-d58e-453d-b4a0-15d5d7e0e51a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:49.808474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 600/1205] 10.222.0.73 () {60 vars in 1132 bytes} [Tue Aug 23 22:13:49 2022] POST /identity/v3/users => generated 251 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:13:51.752386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72a23-1b92-48fa-be80-01db3836aac9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:51.752997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72a23-1b92-48fa-be80-01db3836aac9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:51.752997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d72a23-1b92-48fa-be80-01db3836aac9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:51.754242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 606/1206] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:13:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:51.759569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:51.762702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:51.762702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:51.779194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:51.783812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:51.847374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 601/1207] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:52.123727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d72a23-1b92-48fa-be80-01db3836aac9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.228426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d72a23-1b92-48fa-be80-01db3836aac9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.263174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.264042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.264395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.264671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.274962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:52.280537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:52.286012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:52.287737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 607/1208] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:52.289138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.327877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.386083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5896b4-1e37-4bbb-81b1-7d6083c03f1d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.419956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.420556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.420836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.421235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.421799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:52.422082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.423437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:52.431059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 602/1209] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:52.436920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.488397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.537983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195b498-b97f-49c1-9c89-df45484caeca None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.570780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.571405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.571682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.571953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.583044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.584552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:52.598986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:52.600115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:52.601216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:52.601216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:52.601216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:52.605004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 608/1210] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:52.612071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.705666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.708579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b24c6f-96e8-4ba9-b25c-86e00196964a None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.741996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.742598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.742877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.743148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.743786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:52.744087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.745527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:52.757329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 603/1211] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:52.763682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:52.864062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0e8843-a8cb-4520-8d13-cd15cd0799c4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:52.865646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:52.898023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:52.898644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:52.898922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:52.899192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:52.907634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:52.909118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:52.920955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:52.922160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:52.923318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 609/1212] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:52.928068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.030159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf50bef6-07f0-4866-8f96-0266ce327726 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.063379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.063982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.064278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.064529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.065283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:53.065585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.067029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.084192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 604/1213] 10.222.0.73 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:52 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:53.090039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.191806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4acadda-dac0-4c2a-a0fb-d0e473271203 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.226191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.226872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.227162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.227445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.258100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:53.295722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.297123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.395206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 610/1214] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:13:53.401378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.420737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:53.499805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5920c440-9684-4b30-8c91-c0f17843a470 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.534270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.534898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.535177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.535452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.545701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.552456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:53.562181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:53.568114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:53.569334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:53.569334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:53.569334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:13:53.569334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 605/1215] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:13:53.569662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.630349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:53.669318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e72cd26-c7ac-417f-80a8-3c0f0b84f333 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.704212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.704837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.705127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.705409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.706054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:53.706390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.707757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.719883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 611/1216] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:53.725926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.797608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:53.828163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0758aa-fe8d-4003-9ca2-932eb84929ff None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:53.860677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:53.874960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:53.874960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:53.874960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:53.874960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:53.874960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:53.881085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:53.882162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:13:53.883310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 606/1217] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:53.889319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:53.988788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae1da39-42c0-4060-8655-96e29d546cc7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:54.013607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:54.025167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:54.026026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:54.026307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:54.026578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:54.027312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:54.027616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:54.029066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a28187-6b15-4b85-af9e-650db401ae90 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:54.040244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 612/1218] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:54.047074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:54.146761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd4c458-33e4-45be-bcc2-b342342d243d None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:54.174926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:54.181121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:54.181724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:54.181980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:54.182251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:54.182968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=f077a58c9406414085dba8d67c69e364) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:54.183270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:54.185293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa513e-d9e6-44bc-99fa-5958320bbec4 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:54.190597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 607/1219] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:13:54 2022] GET /identity/v3/role_assignments?user.id=f077a58c9406414085dba8d67c69e364&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:56.104016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d7e39-303b-47b2-8bf8-16fa8ddb5de1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:56.104544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d7e39-303b-47b2-8bf8-16fa8ddb5de1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:56.104544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d7e39-303b-47b2-8bf8-16fa8ddb5de1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:56.105975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 613/1220] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:13:56.111471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:56.111894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:56.112048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:56.130764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:13:56.131948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:13:56.199643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 608/1221] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:13:56 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:13:56.469537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d7e39-303b-47b2-8bf8-16fa8ddb5de1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:56.572393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d7e39-303b-47b2-8bf8-16fa8ddb5de1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:56.605568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:56.606790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:56.607117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:56.607436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:56.617536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:13:56.622981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:56.624030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:56.625322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 614/1222] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:13:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:56.630327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:56.734428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbb6d45-a630-4d6c-964c-9f896481d8c3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:56.757423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:56.767652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:56.768261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:56.768538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:56.768810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:56.769503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] RBAC: Adding query filter params (name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:56.769803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:56.771148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:56.779637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 609/1223] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:13:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:56.785632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:56.829001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:56.884178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3e1531-7684-4d89-90a1-4e082ceb7143 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:56.919933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:56.920563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:56.920887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:56.921459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:56.933589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:56.935230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:56.950849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:56.952086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:56.953415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:56.953415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:56.953415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:56.953415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 615/1224] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:56.959591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.061192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b227ca-5123-4893-a23e-1b3a10413887 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.093322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.093913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.094189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.094453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.095077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:57.095373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:57.096715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:57.108602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 610/1225] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:57.114552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.206973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:57.213470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596cf0e-8ae7-43f9-9eef-6a817a1a568f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.246597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.247959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.248238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.248532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.255616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:57.257243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:13:57.268832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:57.269895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:57.270915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 616/1226] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:57.277875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.332484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:57.384961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3849be-7795-4ee5-9de3-fa0d0bf51e69 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.418199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.419466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.419801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.420011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.420735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:57.421054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:57.422473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:57.439624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 611/1227] 10.222.0.73 () {56 vars in 1143 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:57.445423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.548953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e23de8-79e8-40ce-9571-06c5ff9e93a0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.581733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.582356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.582684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.582907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.593837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:57.595267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:13:57.609819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:57.615061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:13:57.616175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:13:57.616175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:13:57.616175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:57.616175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 617/1228] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:57.617096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.718436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf13ef8f-fdd2-4e18-a495-b6f0f4656178 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.720793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:57.752876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.753505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.753787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.754094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.754735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:57.755038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:57.756402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:57.768228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 612/1229] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:13:57.774158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:57.784665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:57.873130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dc1d6a-0464-4b46-aed1-8bd501328769 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:57.906363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:57.906980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:57.907256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:57.907526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:57.913898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:57.915341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:13:57.926334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:13:57.927416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:13:57.928744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 618/1230] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:13:57.933537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:58.033574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5576ff-7c69-4d05-b29f-1a4c33891aa6 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:58.066822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:58.067411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:58.067688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:58.067955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:58.068629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:13:58.068924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:58.070348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0cf822-2056-499a-99ed-7417c27dd45a None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:58.082097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 613/1231] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:13:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:13:58.088276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:13:58.152568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:13:58.187560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe581d8-afb2-4a8f-898f-bb3810b74bb9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:13:58.221938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c803cf-4511-46d1-916f-4b7c58f5f7a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:13:58.222638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c803cf-4511-46d1-916f-4b7c58f5f7a2 None admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:13:58.222914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c803cf-4511-46d1-916f-4b7c58f5f7a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:13:58.223187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c803cf-4511-46d1-916f-4b7c58f5f7a2 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/f077a58c9406414085dba8d67c69e364/roles/fe14249a99f34d5ab0cf08b3c2bd6b13` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:13:58.237881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c803cf-4511-46d1-916f-4b7c58f5f7a2 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=f077a58c9406414085dba8d67c69e364, role_id=fe14249a99f34d5ab0cf08b3c2bd6b13)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:13:58.239356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c803cf-4511-46d1-916f-4b7c58f5f7a2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:13:58.374806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 619/1232] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:13:58 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/f077a58c9406414085dba8d67c69e364/roles/fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:14:00.303872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48d474-d0d1-4f94-8649-eadd70dfa68c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:00.304419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48d474-d0d1-4f94-8649-eadd70dfa68c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:00.304419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b48d474-d0d1-4f94-8649-eadd70dfa68c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:00.305839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 614/1233] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:00.312034 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:00.312490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:00.312612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:00.331073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:00.332267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:00.398880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 620/1234] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:00.680419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48d474-d0d1-4f94-8649-eadd70dfa68c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:00.706295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:00.785172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b48d474-d0d1-4f94-8649-eadd70dfa68c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:00.822636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:00.822636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:00.822636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:00.822636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:00.829914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:14:00.834592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:00.835721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:00.836615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 615/1235] 10.222.0.73 () {56 vars in 1102 bytes} [Tue Aug 23 22:14:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:00.842404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:00.887899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:00.940038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46f91f-7f96-45d0-8d2d-bde20b1abba8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:00.974378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:00.974984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:00.975267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:00.975540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:00.976274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] RBAC: Adding query filter params (name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:00.976581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:00.977979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:00.985808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 621/1236] 10.222.0.73 () {56 vars in 1103 bytes} [Tue Aug 23 22:14:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:00.991585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:01.078897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:01.094199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c9ce5e-2e8a-4342-9db7-4b16e7d1f903 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:01.127674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:01.128297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:01.128576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:01.128891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:01.139318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:01.140807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:01.155846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:01.156955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:01.157876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:01.157876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:01.157876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:01.157876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 616/1237] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:01.162932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:01.265312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d13fa1-0952-41bd-994f-c77226d38510 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:01.305298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:01.306080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:01.306451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:01.306815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:01.307819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:01.308258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:01.310297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:01.323494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 622/1238] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:01.330858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:01.336864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:01.430984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a7211c-e2eb-4b29-adac-c8727f9be3a3 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:01.463568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:01.464180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:01.464453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:01.464721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:01.467880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:01.472758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:01.474209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:14:01.485418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:01.486587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:01.487554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 617/1239] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:01 2022] GET /identity/v3/users/placement => generated 87 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:14:01.492405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:01.593848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f60c2-de00-4b65-92ea-053e2fc966c4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:01.627081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:01.627680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:01.627960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:01.628227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:01.628949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:01.629355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:01.630759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:01.648455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 623/1240] 10.222.0.73 () {56 vars in 1143 bytes} [Tue Aug 23 22:14:01 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:01.655748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:01.755310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a0d34-1d31-46b5-b838-2b7ac695a3e7 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:01.789299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:01.789996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:01.790279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:01.790553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:01.851389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:01.858100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:01.859721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:01.964296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 618/1241] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:14:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:14:01.970551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:01.997565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:02.070961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aca9e05-75e5-4e00-8d0e-8645dcb39c2c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:02.104849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:02.105436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.105717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.105991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.116186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:02.117678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:02.132745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:02.134017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:02.135322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:02.135322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:02.135322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:02.135322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 624/1242] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:02.140416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:02.234562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:02.240277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835fa2a4-f3f0-4bda-bf8c-35a2f64cb233 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:02.276814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:02.277443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.277716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.277981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.278657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:02.278952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:02.280309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:02.291610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 619/1243] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:02.298247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:02.379116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:02.401205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415de23-9c2b-414a-991b-5e65d0a7ffe4 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:02.434332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:02.435547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.435856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.436165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.442652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:02.448747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:14:02.455771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:02.456834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:02.457998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 625/1244] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:14:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:02.462997 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:02.564329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c12f92a-3d13-4360-8d71-e954bab2c0da None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:02.598625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:02.599232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.599515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.599789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.600475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:02.600784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:02.602243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0f3ea3-1eb8-4fb0-8eb8-a0ded39e7860 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:02.614551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 620/1245] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:14:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:02.620799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:02.723391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ded887-b99b-42a5-a7ca-004ac521e9fa None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:02.757444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:02.758040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:02.758319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:02.758593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:02.759312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] RBAC: Adding query filter params (role.id=fe14249a99f34d5ab0cf08b3c2bd6b13, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=f077a58c9406414085dba8d67c69e364) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:02.759622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:02.761077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d4a2a9-c4e9-45c8-b0a3-a732958e577e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:02.766753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 626/1246] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:14:02 2022] GET /identity/v3/role_assignments?user.id=f077a58c9406414085dba8d67c69e364&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=fe14249a99f34d5ab0cf08b3c2bd6b13 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:04.733320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed7b9-16dc-4edf-bbad-461e591260b0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:04.733914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed7b9-16dc-4edf-bbad-461e591260b0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:04.733914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d33ed7b9-16dc-4edf-bbad-461e591260b0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:04.735260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 621/1247] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:04.740810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:04.741289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:04.741394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:04.748653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:04.760534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:04.761769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:04.827721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 627/1248] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:04 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:05.102140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed7b9-16dc-4edf-bbad-461e591260b0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.121547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.205028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d33ed7b9-16dc-4edf-bbad-461e591260b0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.237813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.238438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.238715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.238982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.250124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:14:05.254843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:05.255941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:05.256811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 622/1249] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:05.262117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.364594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f0fd1-a56a-4359-a987-cafd8cba4cf2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.397027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.402272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.406005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.406005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.406005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.406005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:05.406005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.406436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.414173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 628/1250] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:05.420838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.499793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.522777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2743d2a0-9228-4540-98de-89044deaa88c None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.556773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.557852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.558133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.558407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.569172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.570672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:05.585980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:05.587062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:05.587987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:05.587987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:05.587987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:05.587987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 623/1251] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 23 22:14:05.593473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.693443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88bd1380-b4a3-4a1a-9a97-59ae7837d44e None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.726958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.727562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.727840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.728110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.728746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:05.729070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.730438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.742673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 629/1252] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:05.750503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.851255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799bc297-19fd-4c10-bc9a-848bb62a7626 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:05.878510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:05.886126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:05.886802 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:05.887029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:05.887300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:05.894789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:05.896214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:14:05.907689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:05.908907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:05.910154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 624/1253] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:05.914732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:05.970092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:06.014407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0025fa-4481-4436-8825-97d20a7893a9 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:06.048884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:06.049516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:06.049794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:06.050063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:06.050795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:06.051097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:06.052510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:06.070127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 630/1254] 10.222.0.73 () {56 vars in 1143 bytes} [Tue Aug 23 22:14:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:06.076411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:06.175865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baea7d6-eca3-4836-9249-44f8d6c1f0ce None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:06.208666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:06.209302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:06.209582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:06.209852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:06.265841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:06.277789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:06.279760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:06.377687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 625/1255] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:14:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:14:06.383695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:06.482249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5673a426-c5a4-40d0-abfa-4f341100a26c None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:06.496282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:06.517410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:06.517974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:06.518244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:06.518510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:06.528467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:06.529949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:06.543589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:06.545355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:06.546575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:06.546575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:06.546575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:06.546575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 631/1256] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:06.551313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:06.638737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:06.650644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb34c78-ce99-4770-a698-9d24f3c06c70 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:06.685174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:06.686172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:06.686662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:06.687157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:06.688301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:06.688856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:06.692082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:06.709376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 626/1257] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:06.715217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:06.818084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911876a-658f-434b-9aaa-69a66e0ce466 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:06.851787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:06.852410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:06.852687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:06.852956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:06.860278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:06.861741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:14:06.873031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:06.874074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:06.875315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 632/1258] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:14:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:06.880039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:06.979693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1a4259-cddc-4cdd-837e-94ffe07ef076 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:07.013781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:07.014385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:07.014664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:07.014935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:07.015728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:07.016035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:07.017487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770ec0bb-266f-41e0-a40a-39f9a8ed640f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:07.030036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 627/1259] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:14:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:07.036416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:07.062688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:07.134170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d4a9ab-31b8-4f68-a62b-d7e3d1526336 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:07.167060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:07.167663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:07.167944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:07.168217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:07.169572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=f077a58c9406414085dba8d67c69e364) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:07.169881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:07.171302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c897fc8b-9962-402d-8e51-d624b9ded98f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:07.176727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 633/1260] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:14:07 2022] GET /identity/v3/role_assignments?user.id=f077a58c9406414085dba8d67c69e364&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 259 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:09.063178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf1d09-1af3-41c2-b63d-25760f414d4b None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.063701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf1d09-1af3-41c2-b63d-25760f414d4b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.063701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf1d09-1af3-41c2-b63d-25760f414d4b None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.065077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 628/1261] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:09.070623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.071018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.071173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.089204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:09.090897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:09.158836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 634/1262] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:09.433500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf1d09-1af3-41c2-b63d-25760f414d4b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:09.444711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:09.538830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf1d09-1af3-41c2-b63d-25760f414d4b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:09.572206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:09.572855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.573170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.573445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.584390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:14:09.589246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:09.590978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:09.591900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 629/1263] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:14:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:09.597304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:09.615696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:09.695042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7c04aa-5ab4-4652-ae93-3fa9fc9c1edd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:09.734715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:09.735323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.735664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.735963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.736662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:09.736968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:09.738495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:09.746699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 635/1264] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:14:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:09.752590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:09.820724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:09.853468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de299dd8-2299-4e77-ba05-6a9bba23eea5 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:09.886970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:09.887644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:09.887994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:09.888902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:09.899191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:09.900867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:09.914908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:09.919057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:09.920351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:09.920351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:09.920351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:09.920351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 630/1265] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:09.922277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.002433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:10.020927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b8a3b9-89f8-4f6a-909b-5e7fe2ecd750 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:10.053556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:10.054185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:10.054443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:10.054755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:10.055395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:10.055709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:10.057073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:10.068549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 636/1266] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:14:10.074420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.177389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee2b7d0-4448-4e29-aa9a-7916eed79ac0 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:10.197222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:10.210748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:10.211366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:10.211660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:10.211929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:10.219049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:10.220576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:14:10.231282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:10.232410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:10.234096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 631/1267] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:10 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:10.239037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.340932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c911eba0-37d1-4508-83f4-c45d5c4e0fb7 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:10.374703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:10.375304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:10.375591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:10.375864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:10.376592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:10.376901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:10.378447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:10.387731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:10.395679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 637/1268] 10.222.0.73 () {56 vars in 1143 bytes} [Tue Aug 23 22:14:10 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:10.401795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.506421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb917aec-b92d-441f-b262-736bdc52b750 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:10.542944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:10.543217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:10.543373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:10.543603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:10.555289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:10.556736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:10.571207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:10.578252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:10.579353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:10.579353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:10.579353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:10.579353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 632/1269] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:10.579648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.681094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d007c50-497b-449c-9b0d-18dfb5ffd85d None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:10.715250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:10.715857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:10.716137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:10.716408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:10.717117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:10.717950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:10.719810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:10.731058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 638/1270] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:14:10.737691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.769583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:10.838534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07fbc26-9f0a-430e-a0f0-1ec460e6ab2f None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:10.871704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:10.872334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:10.872607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:10.872875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:10.879466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:10.880916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:14:10.891992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:10.893046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:10.893959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 633/1271] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:14:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:10.899208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:10.958009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:11.003269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e31f267-148d-4c2b-9ede-cc0f27053696 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:11.036889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:11.037523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:11.037802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:11.038104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:11.038785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:11.039085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:11.040494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3e06bd9-fce4-486d-97f7-e3d89eacf015 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:11.054739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 639/1272] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:14:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:11.060875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:11.153086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:11.161981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd18c1-a81d-41ff-bc8f-9bb8e45401ab None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:11.196920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae7e276-2608-445c-a5cf-99feff84a8e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:11.201086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e276-2608-445c-a5cf-99feff84a8e8 None admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:11.201086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e276-2608-445c-a5cf-99feff84a8e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:11.201086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae7e276-2608-445c-a5cf-99feff84a8e8 None admin] PATH_INFO: `/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/f077a58c9406414085dba8d67c69e364/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:11.201086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7e276-2608-445c-a5cf-99feff84a8e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=fed3c0cb3baf419aa351a156ce71854b, user_id=f077a58c9406414085dba8d67c69e364, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:11.201538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae7e276-2608-445c-a5cf-99feff84a8e8 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:11.290442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 634/1273] 10.222.0.73 () {58 vars in 1439 bytes} [Tue Aug 23 22:14:11 2022] PUT /identity/v3/projects/fed3c0cb3baf419aa351a156ce71854b/users/f077a58c9406414085dba8d67c69e364/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:14:13.224578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5bff6-f047-4d6c-a449-6c23f15e67de None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:13.225125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5bff6-f047-4d6c-a449-6c23f15e67de None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:13.225125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad5bff6-f047-4d6c-a449-6c23f15e67de None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:13.226531 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 640/1274] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:13.231761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:13.232167 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:13.232318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:13.252331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:13.253560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:13.320341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 635/1275] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:13 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:13.591637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5bff6-f047-4d6c-a449-6c23f15e67de None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:13.693136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad5bff6-f047-4d6c-a449-6c23f15e67de None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:13.710749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:13.727607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:13.728230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:13.728503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:13.728766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:13.739208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 23 22:14:13.743883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:13.745032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:13.745987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 641/1276] 10.222.0.73 () {56 vars in 1096 bytes} [Tue Aug 23 22:14:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:13.751196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:13.833829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:13.852634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e05d2b-88e5-40c2-b142-ca0e055037ba None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:13.887055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:13.887655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:13.887929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:13.888195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:13.888845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:13.889176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:13.890525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:13.898305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 636/1277] 10.222.0.73 () {56 vars in 1099 bytes} [Tue Aug 23 22:14:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:13.905067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:14.004920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7febd468-62d6-4a70-b2ed-4cdaab1af0c0 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:14.038338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:14.038956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.039233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.039503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.049955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:14.051372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:14.065882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:14.066990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:14.068099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:14.068099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:14.068099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:14.068099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 642/1278] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:14.073029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:14.174148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f61d86-0a09-42eb-98a9-0eb22008bc97 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:14.207931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:14.208598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.208873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.209184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.209799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:14.210095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:14.211434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:14.223053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 637/1279] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:14.229002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:14.260889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:14.329991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac91ad3-d164-49ba-b549-1856b82d0bc1 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:14.363453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:14.364073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.364348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.364618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] PATH_INFO: `/v3/users/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.371412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:14.373170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 23 22:14:14.385537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:14.386663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:14.387869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 643/1280] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:14 2022] GET /identity/v3/users/placement => generated 87 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:14.392834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:14.396688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:14.489961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa74da8-a7ff-41d3-b58e-4bd6bf34235e None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:14.525032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:14.525646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.525924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.526195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.526908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:14.527206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:14.528610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:14.547623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 638/1281] 10.222.0.73 () {56 vars in 1143 bytes} [Tue Aug 23 22:14:14 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:14.553742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:14.642975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:14.652328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80189763-8053-4df4-b557-03b5ccdd72ba None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:14.684847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:14.686528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.686817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.687565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:14.754612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:14.755976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:14.843666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:14.853787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 644/1282] 10.222.0.73 () {58 vars in 1282 bytes} [Tue Aug 23 22:14:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 23 22:14:14.860320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:14.921400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:14.959087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368362c0-5c19-474f-8c1a-b0473d7b7790 None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:14.992349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:14.992967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:14.993277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:14.993548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.005043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.006480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 23 22:14:15.020767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:15.021927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 23 22:14:15.023265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 23 22:14:15.023265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 23 22:14:15.023265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:15.023265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 639/1283] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:14:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:15.027892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.128081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b84b743-c67b-4c57-8d08-1d554d5d5657 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.161982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.162588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.162870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.163143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.163780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:15.164087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.165490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.177215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 645/1284] 10.222.0.73 () {56 vars in 1109 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:15.183172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.283323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a00827-7a7f-41cb-acba-a27d8fa8daab None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.296983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:15.316020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.316638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.316911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.317207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.323557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.324973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 23 22:14:15.336120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:15.337255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: ERROR keystone.server.flask.application Aug 23 22:14:15.338565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 640/1285] 10.222.0.73 () {56 vars in 1111 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:15.344252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.393702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:15.442813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08f0c1e-ca1c-42c1-95b6-97a2fb868a7f None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.476661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.477325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.477606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.477879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.478565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:15.478870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.480314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1ebd0c-1338-4e49-a5d1-cb26104d05ae None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.492181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 646/1286] 10.222.0.73 () {56 vars in 1148 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:15.498344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:15.601405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b911ff8-bf65-4085-ac31-804a04fbcfba None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:15.633776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:15.634366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:15.634637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:15.634901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:15.635610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] RBAC: Adding query filter params (role.id=1e8252b04575454e9d3dc7f3cb1dc64c, scope.project.id=fed3c0cb3baf419aa351a156ce71854b, user.id=f077a58c9406414085dba8d67c69e364) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:15.635910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:15.637438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718b1906-729b-456e-965a-4d86d3c2e0aa None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:15.643679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 641/1287] 10.222.0.73 () {56 vars in 1374 bytes} [Tue Aug 23 22:14:15 2022] GET /identity/v3/role_assignments?user.id=f077a58c9406414085dba8d67c69e364&scope.project.id=fed3c0cb3baf419aa351a156ce71854b&role.id=1e8252b04575454e9d3dc7f3cb1dc64c => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:17.619514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c409-3931-4bdd-91dd-ad9e1587df32 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:17.620053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c409-3931-4bdd-91dd-ad9e1587df32 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:17.620053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044c409-3931-4bdd-91dd-ad9e1587df32 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:17.621445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 647/1288] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:17.633860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:17.633860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:17.633860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:17.649416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:17.651003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:17.684281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:17.723071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 642/1289] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:17.995213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c409-3931-4bdd-91dd-ad9e1587df32 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:18.100186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044c409-3931-4bdd-91dd-ad9e1587df32 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:18.111715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:18.133963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:18.139672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:18.139672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:18.139672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:18.139672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:18.139672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:18.145445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:18.146781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:18.148001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 648/1290] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:14:17 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:18.152983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:18.252733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e80d6e-da4e-4547-b061-a33545c875a8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:18.286779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:18.287392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:18.287674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:18.287945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:18.288625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:18.288930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:18.290374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641022fa-9223-46d8-b60b-a14d0dfb162f None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:18.317187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 643/1291] 10.222.0.73 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:18 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:18.317342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:18.405758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539b6a5-4310-45c1-9e25-a2303edd5073 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:18.439054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:18.460244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:18.460244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:18.460244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] PATH_INFO: `/v3/services` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:18.460244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:18.460244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:18.460244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb645de-d235-45d6-8beb-35744230e8b2 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:18.460831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 649/1292] 10.222.0.73 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:18 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:20.345036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905268c6-e9c6-4b59-9e42-099f834b9a62 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:20.345592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905268c6-e9c6-4b59-9e42-099f834b9a62 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:20.345592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905268c6-e9c6-4b59-9e42-099f834b9a62 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:20.346977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 644/1293] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:20.352262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc997fed-a166-491e-bec8-f2fd2208109a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:20.352724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc997fed-a166-491e-bec8-f2fd2208109a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:20.352824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc997fed-a166-491e-bec8-f2fd2208109a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:20.371648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bc997fed-a166-491e-bec8-f2fd2208109a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:20.372833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc997fed-a166-491e-bec8-f2fd2208109a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:20.407174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:20.454006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 650/1294] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:20 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:20.726167 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905268c6-e9c6-4b59-9e42-099f834b9a62 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:20.756888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:20.829136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905268c6-e9c6-4b59-9e42-099f834b9a62 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:20.862911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b86450-513e-4bbf-a30d-9ec9cb7e4bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:20.863508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b86450-513e-4bbf-a30d-9ec9cb7e4bbf None admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:20.863786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b86450-513e-4bbf-a30d-9ec9cb7e4bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:20.864056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b86450-513e-4bbf-a30d-9ec9cb7e4bbf None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:20.864703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b86450-513e-4bbf-a30d-9ec9cb7e4bbf None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:20.866129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b86450-513e-4bbf-a30d-9ec9cb7e4bbf None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:20.981862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 645/1295] 10.222.0.73 () {60 vars in 1141 bytes} [Tue Aug 23 22:14:20 2022] POST /identity/v3/services => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:14:22.942588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb02ff1-6825-4d83-8d7b-74eef7df3a9d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:22.943164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb02ff1-6825-4d83-8d7b-74eef7df3a9d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:22.943164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb02ff1-6825-4d83-8d7b-74eef7df3a9d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:22.944453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 651/1296] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:22.949930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:22.950367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:22.950468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:22.969409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:22.972540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:23.043972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 646/1297] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:22 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:23.319073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb02ff1-6825-4d83-8d7b-74eef7df3a9d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:23.422613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb02ff1-6825-4d83-8d7b-74eef7df3a9d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:23.457734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:23.458388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:23.458672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:23.458933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:23.459512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:23.460866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:23.469499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:23.470484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:23.471400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 652/1298] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:14:23 2022] GET /identity/v3/services/placement => generated 90 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:23.476752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:23.495271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:23.582023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dfdfbb-7168-4f3d-a64d-ed1c4795d44a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:23.615223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:23.615825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:23.616105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:23.616375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:23.616995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:23.617328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:23.618683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:23.624351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 647/1299] 10.222.0.73 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:23 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:14:23.630471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:23.662337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:23.731423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5394c7-7bac-4a1c-a7ad-6130f7debb47 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:23.766075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:23.766672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:23.766949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:23.767219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:23.767913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] RBAC: Adding query filter params (interface=public, service_id=d14277b8e37a45e2bf4df35f740d30dc, region_id=RegionOne) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:23.768218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:23.769670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b63c76-d778-47b5-9cf5-de6e29b808ba None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:23.774757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 653/1300] 10.222.0.73 () {56 vars in 1251 bytes} [Tue Aug 23 22:14:23 2022] GET /identity/v3/endpoints?service_id=d14277b8e37a45e2bf4df35f740d30dc&interface=public®ion_id=RegionOne => generated 194 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:23.781327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:23.862195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:23.879531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d8f7f1-e9f4-4353-875a-8701a46eabff None admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:23.913639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd4a920-0117-4b05-b01b-1adb069f53da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:23.914236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4a920-0117-4b05-b01b-1adb069f53da None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:23.914515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4a920-0117-4b05-b01b-1adb069f53da None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:23.914786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd4a920-0117-4b05-b01b-1adb069f53da None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:23.915383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4a920-0117-4b05-b01b-1adb069f53da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:23.916734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd4a920-0117-4b05-b01b-1adb069f53da None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:23.923921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 648/1301] 10.222.0.73 () {56 vars in 1087 bytes} [Tue Aug 23 22:14:23 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:14:25.856111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fb2e6-c459-4ef6-8d58-a95fdac28713 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:25.856612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fb2e6-c459-4ef6-8d58-a95fdac28713 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:25.856612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00fb2e6-c459-4ef6-8d58-a95fdac28713 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:25.858037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 654/1302] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:14:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:14:25.863256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:25.863714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:25.863823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:25.882487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:14:25.883667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:14:25.951103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 649/1303] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:14:25 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 23 22:14:26.226071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fb2e6-c459-4ef6-8d58-a95fdac28713 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:26.320115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:26.326758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00fb2e6-c459-4ef6-8d58-a95fdac28713 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:26.360636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:26.361291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:26.361576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:26.361851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:26.362445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:26.363800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 23 22:14:26.372260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 23 22:14:26.373539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.get_or_create( Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application with Lock( Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self._enter() Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application return self.creator() Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application created_value = creator( Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: ERROR keystone.server.flask.application Aug 23 22:14:26.374391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 655/1304] 10.222.0.73 () {56 vars in 1117 bytes} [Tue Aug 23 22:14:26 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 23 22:14:26.379527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:26.455354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:26.479903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05694905-a590-42f1-b3ec-223ce9436b66 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:26.514974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:26.515618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:26.515932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:26.516236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] PATH_INFO: `/v3/services` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:26.516864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:14:26.517202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:26.518558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dbc929-19ce-4516-b955-61c6ce515d43 None admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:26.523678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 650/1305] 10.222.0.73 () {56 vars in 1116 bytes} [Tue Aug 23 22:14:26 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:14:26.530652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:14:26.630583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9e3d03-9581-411f-a406-f6cda9959592 None admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:14:26.664502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d4a0fa-a935-4009-9a88-2575ba7ba8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:14:26.665132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4a0fa-a935-4009-9a88-2575ba7ba8d9 None admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:14:26.665415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4a0fa-a935-4009-9a88-2575ba7ba8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:14:26.665686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d4a0fa-a935-4009-9a88-2575ba7ba8d9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:14:26.666332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4a0fa-a935-4009-9a88-2575ba7ba8d9 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:14:26.667689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d4a0fa-a935-4009-9a88-2575ba7ba8d9 None admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:14:26.790711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 656/1306] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:14:26 2022] POST /identity/v3/endpoints => generated 331 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:14:36.561519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=30 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:14:36.786552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:31.092458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb59f2-cfc7-4bc1-b047-6ddada5a2182 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:31.093647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb59f2-cfc7-4bc1-b047-6ddada5a2182 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:31.094173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcb59f2-cfc7-4bc1-b047-6ddada5a2182 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:31.102423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b8bff-5dbc-475f-b062-e6395508f33d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:31.103465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b8bff-5dbc-475f-b062-e6395508f33d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:31.103968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b8bff-5dbc-475f-b062-e6395508f33d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:31.109248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 651/1307] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:31 2022] GET /identity => generated 270 bytes in 18 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:31.202308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d73b8bff-5dbc-475f-b062-e6395508f33d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:31.204878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d73b8bff-5dbc-475f-b062-e6395508f33d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:31.388913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 657/1308] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:22:31.700099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20198abc-3fb2-415a-9ca2-fa5b26934799 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:31.700849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20198abc-3fb2-415a-9ca2-fa5b26934799 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:31.701491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20198abc-3fb2-415a-9ca2-fa5b26934799 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:31.704547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 652/1309] 10.222.0.73 () {54 vars in 965 bytes} [Tue Aug 23 22:22:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:31.719903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c5ec-2c32-421d-b2d5-8603357643f4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:31.720216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c5ec-2c32-421d-b2d5-8603357643f4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:31.720514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8c5ec-2c32-421d-b2d5-8603357643f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:31.830314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-eed8c5ec-2c32-421d-b2d5-8603357643f4 None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:31.831537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eed8c5ec-2c32-421d-b2d5-8603357643f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:31.859377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:31.930207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 658/1310] 10.222.0.73 () {58 vars in 1062 bytes} [Tue Aug 23 22:22:31 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:31.941297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20198abc-3fb2-415a-9ca2-fa5b26934799 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:31.943121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-20198abc-3fb2-415a-9ca2-fa5b26934799 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:32.117361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20198abc-3fb2-415a-9ca2-fa5b26934799 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:32.231142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:32.231674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:32.231948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:32.232199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:32.233873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:32.298066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:32.365930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:32.368148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4787f2-6c88-4a51-a4b0-021a1c5337ae service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:32.390739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 653/1311] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:34.868180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea28d-66fe-425c-ab0b-b3b1c694eec2 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:34.868786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea28d-66fe-425c-ab0b-b3b1c694eec2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:34.868786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862ea28d-66fe-425c-ab0b-b3b1c694eec2 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:34.870140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 659/1312] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:34.876527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499e40-31df-48fa-8b8b-ad86aff69e6f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:34.877104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499e40-31df-48fa-8b8b-ad86aff69e6f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:34.877104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0499e40-31df-48fa-8b8b-ad86aff69e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:34.965734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c0499e40-31df-48fa-8b8b-ad86aff69e6f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:34.966966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c0499e40-31df-48fa-8b8b-ad86aff69e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:35.078266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 654/1313] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:35.368478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea28d-66fe-425c-ab0b-b3b1c694eec2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:35.370112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-862ea28d-66fe-425c-ab0b-b3b1c694eec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:35.484922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862ea28d-66fe-425c-ab0b-b3b1c694eec2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:35.565387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:35.568091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:35.568694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:35.568991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:35.569333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:35.570938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:35.695993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:35.697497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f07a0-6d9f-4581-b17e-dd9467b019ca service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:35.720002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 660/1314] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:38.390552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3e93-de7a-478b-8645-a401a21bf18e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:38.391115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3e93-de7a-478b-8645-a401a21bf18e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:38.391115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935a3e93-de7a-478b-8645-a401a21bf18e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:38.392475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 655/1315] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:38.399282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5a79-6cf1-4e21-b4f4-88aa0ef17486 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:38.399818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5a79-6cf1-4e21-b4f4-88aa0ef17486 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:38.399818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9a5a79-6cf1-4e21-b4f4-88aa0ef17486 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:38.474414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1b9a5a79-6cf1-4e21-b4f4-88aa0ef17486 None None] MFA Rules not processed for user `0a614963f72e4c8abe5f162fabb45136`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:38.475737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1b9a5a79-6cf1-4e21-b4f4-88aa0ef17486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:38.533529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:38.583997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 661/1316] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:38 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:38.868262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3e93-de7a-478b-8645-a401a21bf18e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:38.869848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-935a3e93-de7a-478b-8645-a401a21bf18e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:38.879824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:38.989905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935a3e93-de7a-478b-8645-a401a21bf18e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:39.065571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:39.066235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:39.066587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:39.066899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:39.068569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:39.134278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:39.183512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:39.184943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93ae3a3-5a4a-4d0c-af41-acd6e78acbaa service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:39.203799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 656/1317] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:38 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:22:42.066727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926abba-d7ef-4bac-9f45-320744e09081 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:42.067276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926abba-d7ef-4bac-9f45-320744e09081 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:42.067374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0926abba-d7ef-4bac-9f45-320744e09081 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:42.069308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 662/1318] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:42.075209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717959b-a632-4a95-8f35-fa8e963d3c45 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:42.075686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717959b-a632-4a95-8f35-fa8e963d3c45 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:42.075797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717959b-a632-4a95-8f35-fa8e963d3c45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:42.150299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c717959b-a632-4a95-8f35-fa8e963d3c45 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:42.151815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c717959b-a632-4a95-8f35-fa8e963d3c45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:42.255783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 657/1319] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:22:42.581243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edada1-fc73-499f-9fe2-1ddd758102c8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:42.581537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edada1-fc73-499f-9fe2-1ddd758102c8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:42.581822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1edada1-fc73-499f-9fe2-1ddd758102c8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:42.583251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 663/1320] 10.222.0.73 () {54 vars in 965 bytes} [Tue Aug 23 22:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:42.591126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95a88-3072-4032-a9aa-ebeb36430d1f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:42.591482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95a88-3072-4032-a9aa-ebeb36430d1f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:42.591764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad95a88-3072-4032-a9aa-ebeb36430d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:42.619455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:42.683873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1ad95a88-3072-4032-a9aa-ebeb36430d1f None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:42.685570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1ad95a88-3072-4032-a9aa-ebeb36430d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:42.771583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 658/1321] 10.222.0.73 () {58 vars in 1062 bytes} [Tue Aug 23 22:22:42 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:42.782889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edada1-fc73-499f-9fe2-1ddd758102c8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:42.784502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1edada1-fc73-499f-9fe2-1ddd758102c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:42.897410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1edada1-fc73-499f-9fe2-1ddd758102c8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:42.973723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:42.974538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:42.974954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:42.975325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:42.977082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:42.985259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:43.094773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:43.096228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee451a9-41d8-4a88-add3-1f81e607f881 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:43.115094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 664/1322] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:22:46.635048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2f6f-2518-41b5-b76d-5f2097e19910 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:46.635589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2f6f-2518-41b5-b76d-5f2097e19910 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:46.635589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fe2f6f-2518-41b5-b76d-5f2097e19910 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:46.636919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 659/1323] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:46.643681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3c619-057a-460f-8c89-baec45b15e72 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:46.644417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3c619-057a-460f-8c89-baec45b15e72 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:46.644417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd3c619-057a-460f-8c89-baec45b15e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:46.726524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fdd3c619-057a-460f-8c89-baec45b15e72 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:46.727777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fdd3c619-057a-460f-8c89-baec45b15e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:46.797620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:46.842254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 665/1324] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:22:47.143898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2f6f-2518-41b5-b76d-5f2097e19910 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:47.147823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-81fe2f6f-2518-41b5-b76d-5f2097e19910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:47.296067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fe2f6f-2518-41b5-b76d-5f2097e19910 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:47.314956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:47.409499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:47.409992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:47.410276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:47.410563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:47.412874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:47.566733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:47.568204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6503b733-c21a-4045-a82d-986481594224 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:47.589625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 660/1325] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 447 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:49.627278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0527-9e32-450e-8f87-24ae38470ea5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:49.628118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0527-9e32-450e-8f87-24ae38470ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:49.628118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502b0527-9e32-450e-8f87-24ae38470ea5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:49.629287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 666/1326] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:49.635137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5a27-86a7-479b-84c9-e3733d711088 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:49.635611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5a27-86a7-479b-84c9-e3733d711088 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:49.635703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72f5a27-86a7-479b-84c9-e3733d711088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:49.688760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:49.709308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b72f5a27-86a7-479b-84c9-e3733d711088 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:49.710538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b72f5a27-86a7-479b-84c9-e3733d711088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:49.820681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 661/1327] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:50.102121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b0527-9e32-450e-8f87-24ae38470ea5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:50.103933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-502b0527-9e32-450e-8f87-24ae38470ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:50.235135 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502b0527-9e32-450e-8f87-24ae38470ea5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:50.325367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:50.325975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:50.326247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:50.326513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:50.328173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:50.359847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:50.455818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:50.457405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5d0b27-a693-4a67-bfd8-2c8aa9c95633 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:50.489805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 667/1328] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:52.583574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ee6ee-c579-4ff8-bceb-578354e75720 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:52.584729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ee6ee-c579-4ff8-bceb-578354e75720 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:52.584729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ee6ee-c579-4ff8-bceb-578354e75720 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:52.586173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 662/1329] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:52.591930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b39a1-8f1d-4472-8205-cc1503c8edb6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:52.592430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b39a1-8f1d-4472-8205-cc1503c8edb6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:52.592525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0b39a1-8f1d-4472-8205-cc1503c8edb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:52.703954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2b0b39a1-8f1d-4472-8205-cc1503c8edb6 None None] MFA Rules not processed for user `0a614963f72e4c8abe5f162fabb45136`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:52.705253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2b0b39a1-8f1d-4472-8205-cc1503c8edb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:52.784467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:52.830065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 668/1330] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:52 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:22:53.118453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ee6ee-c579-4ff8-bceb-578354e75720 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:53.120140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-312ee6ee-c579-4ff8-bceb-578354e75720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:53.261686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ee6ee-c579-4ff8-bceb-578354e75720 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:53.283416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:53.347621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:53.348853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:53.349234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:53.351049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:53.353563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:53.487429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:53.488809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86079807-10cf-4fe7-ac67-4d0d6ff3f47c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:53.510477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 663/1331] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:53 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:56.691008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95940af-6439-49f0-a7f9-33f1dfa7b766 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:56.691617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95940af-6439-49f0-a7f9-33f1dfa7b766 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:56.691617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95940af-6439-49f0-a7f9-33f1dfa7b766 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:56.693610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 669/1332] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:56.700463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96beaf1-4f28-4b0e-8a10-2d72a213156c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:56.701447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96beaf1-4f28-4b0e-8a10-2d72a213156c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:56.701447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96beaf1-4f28-4b0e-8a10-2d72a213156c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:56.788399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:56.791612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d96beaf1-4f28-4b0e-8a10-2d72a213156c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:56.792854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d96beaf1-4f28-4b0e-8a10-2d72a213156c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:56.957251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 664/1333] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:22:57.246727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95940af-6439-49f0-a7f9-33f1dfa7b766 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:22:57.248459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a95940af-6439-49f0-a7f9-33f1dfa7b766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:57.364480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95940af-6439-49f0-a7f9-33f1dfa7b766 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:22:57.448744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:22:57.449405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:57.449684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:57.449948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:57.451607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:57.480974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:57.585345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:22:57.586910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7969f227-8a39-4575-a8d3-09c3e7b541d9 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:22:57.609202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 670/1334] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:22:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:22:59.666576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa651ee-7f4c-455c-a1b1-fd0766b301b4 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:59.667134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa651ee-7f4c-455c-a1b1-fd0766b301b4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:59.667134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa651ee-7f4c-455c-a1b1-fd0766b301b4 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:59.668466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 665/1335] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:22:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:22:59.674570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f54cb-89bc-46f9-821e-2256b5b1095e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:22:59.675083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f54cb-89bc-46f9-821e-2256b5b1095e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:22:59.675176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0f54cb-89bc-46f9-821e-2256b5b1095e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:22:59.749245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fb0f54cb-89bc-46f9-821e-2256b5b1095e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:22:59.750739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fb0f54cb-89bc-46f9-821e-2256b5b1095e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:22:59.832585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:22:59.856687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 671/1336] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:22:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:23:00.165659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa651ee-7f4c-455c-a1b1-fd0766b301b4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:00.167228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8fa651ee-7f4c-455c-a1b1-fd0766b301b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:00.284251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa651ee-7f4c-455c-a1b1-fd0766b301b4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:00.316633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:00.366796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:00.367430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:00.367690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:00.367961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:00.369646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:00.508859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:00.510320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3dea7-8073-4ff7-ac07-723cfcc78af2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:00.542905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 666/1337] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:03.424445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630765-e49e-4973-8ec8-274a864cbd8a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:03.424991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630765-e49e-4973-8ec8-274a864cbd8a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:03.424991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0630765-e49e-4973-8ec8-274a864cbd8a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:03.426452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 672/1338] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:03.432781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898952c-1607-4259-b780-d3532286ed56 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:03.433298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898952c-1607-4259-b780-d3532286ed56 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:03.433462 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c898952c-1607-4259-b780-d3532286ed56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:03.510154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c898952c-1607-4259-b780-d3532286ed56 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:03.511420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c898952c-1607-4259-b780-d3532286ed56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:03.520298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:03.617500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 667/1339] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:03.906763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630765-e49e-4973-8ec8-274a864cbd8a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:03.908494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a0630765-e49e-4973-8ec8-274a864cbd8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:04.026914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0630765-e49e-4973-8ec8-274a864cbd8a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:04.108760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:04.109413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:04.109692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:04.109962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:04.111593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:04.151177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:04.232586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:04.234089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-851178a3-185c-4c4a-8867-aee0128d13b6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:04.254160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 673/1340] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:08.799947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422bc29-76a3-4c03-a0cf-7cd2393bd10f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:08.800515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422bc29-76a3-4c03-a0cf-7cd2393bd10f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:08.800733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3422bc29-76a3-4c03-a0cf-7cd2393bd10f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:08.802426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 668/1341] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:23:08.811799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f668-9f45-4ba8-a236-bb9660696d53 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:08.812371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f668-9f45-4ba8-a236-bb9660696d53 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:08.814178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f668-9f45-4ba8-a236-bb9660696d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:08.921508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f809f668-9f45-4ba8-a236-bb9660696d53 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:08.922763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f809f668-9f45-4ba8-a236-bb9660696d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:09.060717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:09.075894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 674/1342] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:09.405647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422bc29-76a3-4c03-a0cf-7cd2393bd10f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:09.408458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3422bc29-76a3-4c03-a0cf-7cd2393bd10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:09.534734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3422bc29-76a3-4c03-a0cf-7cd2393bd10f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:09.593612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:09.636189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:09.636857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:09.637187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:09.637464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:09.639201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:09.780741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:09.783722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef8b393-e297-436d-9815-eec2fa493c1b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:09.812630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 669/1343] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:12.812241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeca378-092b-4077-81e2-2dd450cfaeec None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:12.812768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeca378-092b-4077-81e2-2dd450cfaeec None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:12.812768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeca378-092b-4077-81e2-2dd450cfaeec None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:12.814213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 675/1344] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:12.819303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67f466-fa40-409e-9269-9f62ce2d975b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:12.819774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67f466-fa40-409e-9269-9f62ce2d975b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:12.819868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd67f466-fa40-409e-9269-9f62ce2d975b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:12.892624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bd67f466-fa40-409e-9269-9f62ce2d975b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:12.893876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bd67f466-fa40-409e-9269-9f62ce2d975b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:12.912829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:12.995910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 670/1345] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:13.284042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeca378-092b-4077-81e2-2dd450cfaeec None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:13.288404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aeeca378-092b-4077-81e2-2dd450cfaeec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:13.406175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeca378-092b-4077-81e2-2dd450cfaeec None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:13.487559 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:13.488174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:13.488459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:13.488750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:13.491153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:13.537788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:13.607893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:13.609438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4430d3-d4b3-4a3b-8fe3-cdf2942b55ee service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:13.630065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 676/1346] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:13 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:18.318955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779cb3d-e96a-4d5c-b08b-ae9c83920155 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:18.319508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779cb3d-e96a-4d5c-b08b-ae9c83920155 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:18.319508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8779cb3d-e96a-4d5c-b08b-ae9c83920155 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:18.321108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 671/1347] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:18.326744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b760e-fd59-4e97-9078-5d389cb1858c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:18.327226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b760e-fd59-4e97-9078-5d389cb1858c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:18.327341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7b760e-fd59-4e97-9078-5d389cb1858c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:18.403132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5b7b760e-fd59-4e97-9078-5d389cb1858c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:18.404730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5b7b760e-fd59-4e97-9078-5d389cb1858c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:18.518812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:18.522509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 677/1348] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:18.818108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779cb3d-e96a-4d5c-b08b-ae9c83920155 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:18.819666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8779cb3d-e96a-4d5c-b08b-ae9c83920155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:18.936362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8779cb3d-e96a-4d5c-b08b-ae9c83920155 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:18.995326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:19.022950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:19.023561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:19.023842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:19.024111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:19.025801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:19.144467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:19.145980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38229d2b-6653-45ef-b3ee-be14f38c85d9 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:19.166233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 672/1349] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:24.227746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a8cad-89a4-42c5-8b2a-11c5d2e8d501 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:24.228228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a8cad-89a4-42c5-8b2a-11c5d2e8d501 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:24.228337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187a8cad-89a4-42c5-8b2a-11c5d2e8d501 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:24.232680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 678/1350] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:23:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:24.236656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e3171-e665-4348-90ca-125e6813af86 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:24.237149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e3171-e665-4348-90ca-125e6813af86 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:24.237430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0e3171-e665-4348-90ca-125e6813af86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:24.347810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fd0e3171-e665-4348-90ca-125e6813af86 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:24.349241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fd0e3171-e665-4348-90ca-125e6813af86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:24.375688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:24.518916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 673/1351] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:23:24 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:29.027155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb11ea-bea7-4bc9-b5f1-cdf5fbfe9890 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:29.027664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb11ea-bea7-4bc9-b5f1-cdf5fbfe9890 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:29.027744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb11ea-bea7-4bc9-b5f1-cdf5fbfe9890 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:29.029357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 679/1352] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:23:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:29.034560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b49f6-0a02-461f-9a8f-43e091f815d8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:29.036222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b49f6-0a02-461f-9a8f-43e091f815d8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:29.036618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b49f6-0a02-461f-9a8f-43e091f815d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:29.123615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e03b49f6-0a02-461f-9a8f-43e091f815d8 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:29.125261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e03b49f6-0a02-461f-9a8f-43e091f815d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:29.207861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:29.249743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 674/1353] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:23:29 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:30.634844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da347e-8a7e-492e-9313-ab45b7577478 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:30.635573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da347e-8a7e-492e-9313-ab45b7577478 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:30.635573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0da347e-8a7e-492e-9313-ab45b7577478 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:30.636742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 680/1354] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:30.642081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743102f-7803-47b9-9cee-de01b4edb4c1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:30.642602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743102f-7803-47b9-9cee-de01b4edb4c1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:30.642602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743102f-7803-47b9-9cee-de01b4edb4c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:30.726568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9743102f-7803-47b9-9cee-de01b4edb4c1 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:30.727821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9743102f-7803-47b9-9cee-de01b4edb4c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:30.837191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 675/1355] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:23:30 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:34.344014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93debb62-4f7a-42d6-8328-036111aa39ab None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:34.344594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93debb62-4f7a-42d6-8328-036111aa39ab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:34.344594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93debb62-4f7a-42d6-8328-036111aa39ab None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:34.345931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 681/1356] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:34.352411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6ddf9-d9d4-4cd0-aaf2-e28eb4c69504 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:34.352961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6ddf9-d9d4-4cd0-aaf2-e28eb4c69504 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:34.353105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa6ddf9-d9d4-4cd0-aaf2-e28eb4c69504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:34.391819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:34.437203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2fa6ddf9-d9d4-4cd0-aaf2-e28eb4c69504 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:34.439857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2fa6ddf9-d9d4-4cd0-aaf2-e28eb4c69504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:34.603912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 676/1357] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:34.918250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc560075-e526-4a66-99a0-010ce21da979 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:34.919122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc560075-e526-4a66-99a0-010ce21da979 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:34.919904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc560075-e526-4a66-99a0-010ce21da979 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:34.924000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 682/1358] 10.222.0.73 () {54 vars in 962 bytes} [Tue Aug 23 22:23:34 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:34.934582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154bebd-8268-4e5d-9b77-f7035bb846d7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:34.934806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154bebd-8268-4e5d-9b77-f7035bb846d7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:34.935010 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154bebd-8268-4e5d-9b77-f7035bb846d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:35.029353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e154bebd-8268-4e5d-9b77-f7035bb846d7 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:35.030563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e154bebd-8268-4e5d-9b77-f7035bb846d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:35.093764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:35.187053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 677/1359] 10.222.0.73 () {58 vars in 1059 bytes} [Tue Aug 23 22:23:34 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:35.197002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc560075-e526-4a66-99a0-010ce21da979 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:35.198587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bc560075-e526-4a66-99a0-010ce21da979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:35.369880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc560075-e526-4a66-99a0-010ce21da979 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:35.464506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:35.512710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:35.513354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:35.513638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:35.514082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:35.515521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:35.620657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:35.622723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5092f01a-29df-434c-9caf-a9c8815bd426 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:35.644776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 683/1360] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:37.851407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf28f-a55d-45f9-9c27-e6b020553101 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:37.851923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf28f-a55d-45f9-9c27-e6b020553101 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:37.851923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49cf28f-a55d-45f9-9c27-e6b020553101 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:37.853378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 678/1361] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:37.858740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5461be-7fab-495f-b4da-4b37e07bb22e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:37.859336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5461be-7fab-495f-b4da-4b37e07bb22e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:37.859336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5461be-7fab-495f-b4da-4b37e07bb22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:37.932322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7f5461be-7fab-495f-b4da-4b37e07bb22e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:37.933579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f5461be-7fab-495f-b4da-4b37e07bb22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:37.975353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:38.047398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 684/1362] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:23:38.343456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf28f-a55d-45f9-9c27-e6b020553101 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:38.345229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f49cf28f-a55d-45f9-9c27-e6b020553101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:38.506248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49cf28f-a55d-45f9-9c27-e6b020553101 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:38.516258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:38.632325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:38.632948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:38.633275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:38.633551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:38.635273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:38.735740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:38.737665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d1b482-dfd1-4a2d-99f4-2c31d18654ac service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:38.756501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 679/1363] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:41.222455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a5ad7-8f3e-45d0-b58d-6c99448a8c4c None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:41.223001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a5ad7-8f3e-45d0-b58d-6c99448a8c4c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:41.223100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97a5ad7-8f3e-45d0-b58d-6c99448a8c4c None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:41.224436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 685/1364] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:41.230170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7fbb3-a843-4094-8902-cbab2df70981 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:41.230613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7fbb3-a843-4094-8902-cbab2df70981 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:41.238125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf7fbb3-a843-4094-8902-cbab2df70981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:41.247948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:41.303107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aaf7fbb3-a843-4094-8902-cbab2df70981 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:41.304337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aaf7fbb3-a843-4094-8902-cbab2df70981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:41.415098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 680/1365] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:41.710082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0f5b8-7b30-45a3-a71d-d8d6bdb54df5 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:41.710581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0f5b8-7b30-45a3-a71d-d8d6bdb54df5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:41.710986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb0f5b8-7b30-45a3-a71d-d8d6bdb54df5 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:41.712564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 686/1366] 10.222.0.73 () {54 vars in 962 bytes} [Tue Aug 23 22:23:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:41.718402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91f43-330d-4c8e-8402-9a0ff0a62a5b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:41.718691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91f43-330d-4c8e-8402-9a0ff0a62a5b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:41.718969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a91f43-330d-4c8e-8402-9a0ff0a62a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:41.804665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-72a91f43-330d-4c8e-8402-9a0ff0a62a5b None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:41.805952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-72a91f43-330d-4c8e-8402-9a0ff0a62a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:41.847632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:41.941474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 681/1367] 10.222.0.73 () {58 vars in 1059 bytes} [Tue Aug 23 22:23:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:41.949351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0f5b8-7b30-45a3-a71d-d8d6bdb54df5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:41.950978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9fb0f5b8-7b30-45a3-a71d-d8d6bdb54df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:42.125455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb0f5b8-7b30-45a3-a71d-d8d6bdb54df5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:42.150644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:42.288456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:42.289877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:42.290367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:42.290891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:42.292787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:42.392183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:42.394427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17c0fe-6cc5-4e4e-8697-e2748a4068da service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:42.413625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 687/1368] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:23:42.832671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da898750-5b95-42bf-882b-2ec25fde7f78 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:42.833354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da898750-5b95-42bf-882b-2ec25fde7f78 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:42.833354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da898750-5b95-42bf-882b-2ec25fde7f78 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:42.834916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 682/1369] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:42.840703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917c573-1993-485d-b727-0db8aef4d774 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:42.841002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917c573-1993-485d-b727-0db8aef4d774 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:42.841289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917c573-1993-485d-b727-0db8aef4d774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:42.881412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:42.929080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4917c573-1993-485d-b727-0db8aef4d774 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:42.930427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4917c573-1993-485d-b727-0db8aef4d774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:43.051018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 688/1370] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:43.705077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b3e4-29d4-4773-b82c-ab3f322bfb4a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:43.705541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b3e4-29d4-4773-b82c-ab3f322bfb4a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:43.706315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd19b3e4-29d4-4773-b82c-ab3f322bfb4a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:43.709082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 683/1371] 10.222.0.73 () {54 vars in 964 bytes} [Tue Aug 23 22:23:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:43.718084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bd8a0-b4f4-43c6-b002-629a5cf1b2e1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:43.718577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bd8a0-b4f4-43c6-b002-629a5cf1b2e1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:43.719185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bd8a0-b4f4-43c6-b002-629a5cf1b2e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:43.840658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bd1bd8a0-b4f4-43c6-b002-629a5cf1b2e1 None None] MFA Rules not processed for user `f17bae22fe2b4aa09619bdc0b3e0b4a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:43.842243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd1bd8a0-b4f4-43c6-b002-629a5cf1b2e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:43.907138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:43.958392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 689/1372] 10.222.0.73 () {58 vars in 1061 bytes} [Tue Aug 23 22:23:43 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:43.962870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b3e4-29d4-4773-b82c-ab3f322bfb4a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:43.965724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fd19b3e4-29d4-4773-b82c-ab3f322bfb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:44.078155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd19b3e4-29d4-4773-b82c-ab3f322bfb4a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:44.126708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:44.178703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:44.179311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:44.179585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:44.179847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:44.181549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:44.309694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:44.311292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef7c101-924d-477c-b6de-073807161a84 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:44.335772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 684/1373] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:44.770719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f284f-53d4-4f34-8bbb-5af9528458b8 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:44.771306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f284f-53d4-4f34-8bbb-5af9528458b8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:44.771306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f284f-53d4-4f34-8bbb-5af9528458b8 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:44.772687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 690/1374] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:44.778385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0373c-cbf3-4c97-b746-c99297f93283 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:44.778917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0373c-cbf3-4c97-b746-c99297f93283 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:44.779259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b0373c-cbf3-4c97-b746-c99297f93283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:44.849022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-35b0373c-cbf3-4c97-b746-c99297f93283 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:44.850246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-35b0373c-cbf3-4c97-b746-c99297f93283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:44.872039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:44.958053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 685/1375] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:45.254419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f284f-53d4-4f34-8bbb-5af9528458b8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:45.255930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1e4f284f-53d4-4f34-8bbb-5af9528458b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:45.412206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f284f-53d4-4f34-8bbb-5af9528458b8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:45.467077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:45.531311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:45.531788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:45.532067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:45.532331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:45.534610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:45.629238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:45.630878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9cd343-8a94-461a-9ed4-186184b17664 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:45.649177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 691/1376] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:23:47.804198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a431a-570d-4b23-8db6-24a57b37012f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:47.804718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a431a-570d-4b23-8db6-24a57b37012f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:47.804718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363a431a-570d-4b23-8db6-24a57b37012f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:47.806262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 686/1377] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:47.811380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99c69b-a12f-4757-be30-01b6f9970944 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:47.811899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99c69b-a12f-4757-be30-01b6f9970944 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:47.811992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd99c69b-a12f-4757-be30-01b6f9970944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:47.880521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dd99c69b-a12f-4757-be30-01b6f9970944 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:47.881754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd99c69b-a12f-4757-be30-01b6f9970944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:47.888473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:47.988117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 692/1378] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:48.317986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a431a-570d-4b23-8db6-24a57b37012f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:48.319674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-363a431a-570d-4b23-8db6-24a57b37012f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:48.499584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363a431a-570d-4b23-8db6-24a57b37012f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:48.504178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:48.630809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:48.631803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:48.632293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:48.632818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:48.634701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:48.739039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:48.741036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf90822-d4cc-4830-a09a-17f20c092e6d service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:48.762328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 687/1379] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:50.977724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa978aeb-f074-4540-88e7-df2b0274897f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:50.978273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa978aeb-f074-4540-88e7-df2b0274897f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:50.978273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa978aeb-f074-4540-88e7-df2b0274897f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:50.979621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 693/1380] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:50.985212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bcc6a-2716-4e6d-8cb6-0b82a2eb9a0d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:50.985730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bcc6a-2716-4e6d-8cb6-0b82a2eb9a0d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:50.985812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1bcc6a-2716-4e6d-8cb6-0b82a2eb9a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:50.999507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:51.073072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ab1bcc6a-2716-4e6d-8cb6-0b82a2eb9a0d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:51.074186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ab1bcc6a-2716-4e6d-8cb6-0b82a2eb9a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:51.183373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 688/1381] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:51.487354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa978aeb-f074-4540-88e7-df2b0274897f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:51.488050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fa978aeb-f074-4540-88e7-df2b0274897f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:51.646775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:51.658028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa978aeb-f074-4540-88e7-df2b0274897f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:51.804239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:51.805752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:51.806258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:51.806754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:51.809743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:51.973243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:51.976328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb7fbf0-3826-4bb0-8625-98ef4ff0f6aa service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:51.994833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:52.009636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 694/1382] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:54.212502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfbd9c-09dd-4931-b56b-55e18bc4c835 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:54.213109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfbd9c-09dd-4931-b56b-55e18bc4c835 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:54.213109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebfbd9c-09dd-4931-b56b-55e18bc4c835 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:54.214424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 689/1383] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:54.220287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1be7a5-980c-4778-b2be-603c9425f2f9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:54.220881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1be7a5-980c-4778-b2be-603c9425f2f9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:54.220881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1be7a5-980c-4778-b2be-603c9425f2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:54.297112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5c1be7a5-980c-4778-b2be-603c9425f2f9 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:54.298415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c1be7a5-980c-4778-b2be-603c9425f2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:54.428385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 695/1384] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:54.718033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfbd9c-09dd-4931-b56b-55e18bc4c835 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:54.720285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8ebfbd9c-09dd-4931-b56b-55e18bc4c835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:54.827407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:54.907631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebfbd9c-09dd-4931-b56b-55e18bc4c835 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:55.069879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:55.070518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:55.070818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:55.071099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:55.072838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:55.135603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:55.195498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:55.197172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816cbeca-b8d5-482c-a6ad-d03d71fd5d54 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:55.219462 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 690/1385] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:57.657297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741604-f84c-4f34-befb-7eb4f51e6d27 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:57.657855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741604-f84c-4f34-befb-7eb4f51e6d27 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:57.657855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87741604-f84c-4f34-befb-7eb4f51e6d27 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:57.659400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 696/1386] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:57.666569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c915d-d75a-414a-8fd0-24dd1fcba2e3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:57.667062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c915d-d75a-414a-8fd0-24dd1fcba2e3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:57.667414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4c915d-d75a-414a-8fd0-24dd1fcba2e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:57.758634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4d4c915d-d75a-414a-8fd0-24dd1fcba2e3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:57.760285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4d4c915d-d75a-414a-8fd0-24dd1fcba2e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:57.876822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 691/1387] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:23:58.177715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741604-f84c-4f34-befb-7eb4f51e6d27 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:23:58.179347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87741604-f84c-4f34-befb-7eb4f51e6d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:58.271315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:58.354447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87741604-f84c-4f34-befb-7eb4f51e6d27 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:23:58.473808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:23:58.474422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:58.474698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:58.474967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:58.476592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:23:58.518776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:23:58.573987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:23:58.575659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd0d7c6-adfb-4cec-b2b4-d0bf543d6f84 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:23:58.595128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 697/1388] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:23:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:23:59.886011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d5dc9-ca69-4f07-9908-ce3ad15cf8b2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:59.886551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d5dc9-ca69-4f07-9908-ce3ad15cf8b2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:59.886551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d5dc9-ca69-4f07-9908-ce3ad15cf8b2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:59.887961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 692/1389] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:23:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:23:59.896913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726ac5-b216-44e4-96f4-703f20666209 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:23:59.897449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726ac5-b216-44e4-96f4-703f20666209 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:23:59.897586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75726ac5-b216-44e4-96f4-703f20666209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:23:59.968742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-75726ac5-b216-44e4-96f4-703f20666209 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:23:59.970020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-75726ac5-b216-44e4-96f4-703f20666209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:00.069584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 698/1390] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:23:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:00.378965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d8e52-feb3-436c-81e2-85fdbfb2f01a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:00.379227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d8e52-feb3-436c-81e2-85fdbfb2f01a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:00.379536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23d8e52-feb3-436c-81e2-85fdbfb2f01a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:00.380879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 693/1391] 10.222.0.73 () {54 vars in 967 bytes} [Tue Aug 23 22:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:00.386183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ce70-3e4e-458a-b3f9-109272aa2fce None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:00.386767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ce70-3e4e-458a-b3f9-109272aa2fce None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:00.386973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b3ce70-3e4e-458a-b3f9-109272aa2fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:00.403941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:00.472583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-80b3ce70-3e4e-458a-b3f9-109272aa2fce None None] MFA Rules not processed for user `c498027e07ce4bc28bc0518f22047b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:00.473911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80b3ce70-3e4e-458a-b3f9-109272aa2fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:00.561385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 699/1392] 10.222.0.73 () {58 vars in 1064 bytes} [Tue Aug 23 22:24:00 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:00.564053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d8e52-feb3-436c-81e2-85fdbfb2f01a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:00.566171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f23d8e52-feb3-436c-81e2-85fdbfb2f01a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:00.600796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:00.686786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23d8e52-feb3-436c-81e2-85fdbfb2f01a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:00.736320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae5c9-e6f8-4e25-9922-0d3f00e1a5c1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:00.736604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae5c9-e6f8-4e25-9922-0d3f00e1a5c1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:00.737079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8ae5c9-e6f8-4e25-9922-0d3f00e1a5c1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:00.738484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 700/1393] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:24:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:00.744076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d255224-73cc-4fa5-9c5a-c104980cd242 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:00.744370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d255224-73cc-4fa5-9c5a-c104980cd242 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:00.744646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d255224-73cc-4fa5-9c5a-c104980cd242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:00.772942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62de656-d211-4684-8886-065cf11677dc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:00.773618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62de656-d211-4684-8886-065cf11677dc service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:00.774030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62de656-d211-4684-8886-065cf11677dc service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:00.774214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62de656-d211-4684-8886-065cf11677dc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:00.775888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a62de656-d211-4684-8886-065cf11677dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:00.825859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d255224-73cc-4fa5-9c5a-c104980cd242 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:00.827138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d255224-73cc-4fa5-9c5a-c104980cd242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:00.862032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:00.896142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:00.898113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62de656-d211-4684-8886-065cf11677dc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:00.900096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62de656-d211-4684-8886-065cf11677dc service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:00.920239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 694/1394] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:00.943182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 701/1395] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:24:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:01.235043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62de656-d211-4684-8886-065cf11677dc service glance] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:01.236915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a62de656-d211-4684-8886-065cf11677dc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:01.339752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62de656-d211-4684-8886-065cf11677dc service glance] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:01.428685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:01.463150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:01.463757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:01.464038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:01.464313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:01.466902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:01.562838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:01.564558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2319ccc-ba1a-403f-8108-9d6daf2a542d service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:01.583762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 695/1396] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:01.835214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a88f0-326c-4835-b13d-5b580305f4d5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:01.835214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a88f0-326c-4835-b13d-5b580305f4d5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:01.835214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7a88f0-326c-4835-b13d-5b580305f4d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:01.912923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bf7a88f0-326c-4835-b13d-5b580305f4d5 None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:01.914240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bf7a88f0-326c-4835-b13d-5b580305f4d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:02.009134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 702/1397] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:24:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:02.059025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d72d6-5fee-4c26-8a95-9caef2ebf1fb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:02.059575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d72d6-5fee-4c26-8a95-9caef2ebf1fb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:02.059715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-211d72d6-5fee-4c26-8a95-9caef2ebf1fb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:02.061136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 696/1398] 10.222.0.73 () {54 vars in 943 bytes} [Tue Aug 23 22:24:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:02.068487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c3240-34d4-49f5-9664-8b7af1d9b35e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:02.068800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c3240-34d4-49f5-9664-8b7af1d9b35e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:02.069109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836c3240-34d4-49f5-9664-8b7af1d9b35e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:02.143665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:02.164044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-836c3240-34d4-49f5-9664-8b7af1d9b35e None None] MFA Rules not processed for user `a54377901631412abe887223a71d3623`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:02.185066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-836c3240-34d4-49f5-9664-8b7af1d9b35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:02.285521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 703/1399] 10.222.0.73 () {58 vars in 1040 bytes} [Tue Aug 23 22:24:02 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:02.295111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d72d6-5fee-4c26-8a95-9caef2ebf1fb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:02.296674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-211d72d6-5fee-4c26-8a95-9caef2ebf1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:02.393852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:02.443872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-211d72d6-5fee-4c26-8a95-9caef2ebf1fb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:02.551575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:02.552113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:02.552395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:02.552668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:02.554399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:02.631264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:02.640208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:02.646886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06d3635-331a-43c5-aa1e-3376d0b87fa9 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:02.650719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 697/1400] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:24:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:24:03.758937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b60e1-e1ea-47d9-a308-c6b8a907bca4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:03.759851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b60e1-e1ea-47d9-a308-c6b8a907bca4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:03.760247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4b60e1-e1ea-47d9-a308-c6b8a907bca4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:03.762019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 704/1401] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:24:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:03.768051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bf4c6-a1ec-40cd-8ce0-3d96783d08d7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:03.768563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bf4c6-a1ec-40cd-8ce0-3d96783d08d7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:03.768668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4bf4c6-a1ec-40cd-8ce0-3d96783d08d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:03.849472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3e4bf4c6-a1ec-40cd-8ce0-3d96783d08d7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:03.852473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3e4bf4c6-a1ec-40cd-8ce0-3d96783d08d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:03.989112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 698/1402] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:24:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:04.325472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b60e1-e1ea-47d9-a308-c6b8a907bca4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:04.325472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb4b60e1-e1ea-47d9-a308-c6b8a907bca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:04.420314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:04.471603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4b60e1-e1ea-47d9-a308-c6b8a907bca4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:04.604489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:04.612657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:04.612657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:04.612657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:04.612657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:04.713940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:04.723676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa08050-85c4-44f2-a0ce-d51341f1382d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:04.747447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 705/1403] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 471 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:04.859076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:07.013096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d2461-206f-49aa-bfb1-e9b40cd06abb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:07.013680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d2461-206f-49aa-bfb1-e9b40cd06abb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:07.013680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d2461-206f-49aa-bfb1-e9b40cd06abb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:07.015152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 699/1404] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:24:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:07.021413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3aca9-bc03-4dbc-89f0-1ac1b89f9a8e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:07.021967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3aca9-bc03-4dbc-89f0-1ac1b89f9a8e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:07.021967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf3aca9-bc03-4dbc-89f0-1ac1b89f9a8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:07.096615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2bf3aca9-bc03-4dbc-89f0-1ac1b89f9a8e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:07.097414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2bf3aca9-bc03-4dbc-89f0-1ac1b89f9a8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:07.214116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 706/1405] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:07.513353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d2461-206f-49aa-bfb1-e9b40cd06abb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:07.514899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e9d2461-206f-49aa-bfb1-e9b40cd06abb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:07.605440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:07.701884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d2461-206f-49aa-bfb1-e9b40cd06abb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:07.837532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:07.838165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:07.838518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:07.838859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:07.840585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:07.882152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:07.948518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:07.950311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd01562a-aed9-41d6-85c3-2176cd2402d3 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:07.985210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 700/1406] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:10.095925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d49d27-61db-42ab-9650-5fd3e7a91113 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:10.096842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d49d27-61db-42ab-9650-5fd3e7a91113 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:10.096842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d49d27-61db-42ab-9650-5fd3e7a91113 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:10.098200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 707/1407] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:24:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:24:10.105588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d9f0-61ed-4ec0-94c6-c82b0d4343f7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:10.106537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d9f0-61ed-4ec0-94c6-c82b0d4343f7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:10.107013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb3d9f0-61ed-4ec0-94c6-c82b0d4343f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:10.211730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6eb3d9f0-61ed-4ec0-94c6-c82b0d4343f7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:10.219631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6eb3d9f0-61ed-4ec0-94c6-c82b0d4343f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:10.367998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 701/1408] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:10.700172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d49d27-61db-42ab-9650-5fd3e7a91113 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:10.701968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57d49d27-61db-42ab-9650-5fd3e7a91113 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:10.769511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:10.924369 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d49d27-61db-42ab-9650-5fd3e7a91113 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:11.096323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:11.097924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:11.098210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:11.098488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:11.100158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:11.110046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:11.202046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:11.203630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989f3c9e-34f8-4dc9-9d5a-1fb37acae584 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:11.224423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 708/1409] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 524 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:20.365769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:21.221679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=41 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:26.610270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ad88-a32c-42c0-90cd-a5c884f3834c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:26.610887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ad88-a32c-42c0-90cd-a5c884f3834c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:26.610887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb2ad88-a32c-42c0-90cd-a5c884f3834c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:26.612178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 702/1410] 10.222.0.73 () {54 vars in 918 bytes} [Tue Aug 23 22:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:26.617783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32735d1-eb4c-48be-9677-dcdc02d39d51 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:26.618238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32735d1-eb4c-48be-9677-dcdc02d39d51 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:26.618358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32735d1-eb4c-48be-9677-dcdc02d39d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:26.697439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e32735d1-eb4c-48be-9677-dcdc02d39d51 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:26.698703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e32735d1-eb4c-48be-9677-dcdc02d39d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:26.813846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 709/1411] 10.222.0.73 () {58 vars in 1015 bytes} [Tue Aug 23 22:24:26 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:26.828350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e71a-f4f6-49fb-ad9a-e53bb6e9c445 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:26.829293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e71a-f4f6-49fb-ad9a-e53bb6e9c445 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:26.829826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a7e71a-f4f6-49fb-ad9a-e53bb6e9c445 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:26.831281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 703/1412] 10.222.0.73 () {54 vars in 918 bytes} [Tue Aug 23 22:24:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:26.836517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c280fd-10f9-4557-aadb-7c18e8144ef9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:26.836809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c280fd-10f9-4557-aadb-7c18e8144ef9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:26.837144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c280fd-10f9-4557-aadb-7c18e8144ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:26.915615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:26.918405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b6c280fd-10f9-4557-aadb-7c18e8144ef9 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:26.919789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6c280fd-10f9-4557-aadb-7c18e8144ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:27.046736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 710/1413] 10.222.0.73 () {58 vars in 1015 bytes} [Tue Aug 23 22:24:26 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:29.538127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91b76-46b8-4f8f-8a21-5bddd96f9f89 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:29.538719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91b76-46b8-4f8f-8a21-5bddd96f9f89 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:29.538719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab91b76-46b8-4f8f-8a21-5bddd96f9f89 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:29.539987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:29.545495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6d71b-cbf0-476b-96c5-bb55decd20ee None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:29.545946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6d71b-cbf0-476b-96c5-bb55decd20ee None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:29.546050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a6d71b-cbf0-476b-96c5-bb55decd20ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:29.613723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e1a6d71b-cbf0-476b-96c5-bb55decd20ee None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:29.615384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1a6d71b-cbf0-476b-96c5-bb55decd20ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:29.670505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:29.715395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:30.014067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91b76-46b8-4f8f-8a21-5bddd96f9f89 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:30.015602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0ab91b76-46b8-4f8f-8a21-5bddd96f9f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:30.173974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab91b76-46b8-4f8f-8a21-5bddd96f9f89 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:30.251528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:30.292984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:30.293622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:30.293901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:30.294172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:30.295781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:30.387901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:30.389562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce46d7-dd15-4991-ba44-ff1297de34e9 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:30.409986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 705/1416] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:31.325712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443434c-0634-4fc8-ae82-90e77187fb5d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:31.326289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443434c-0634-4fc8-ae82-90e77187fb5d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:31.326289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443434c-0634-4fc8-ae82-90e77187fb5d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:31.327579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:31.333418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b4a19-9205-42d1-9b0c-4f0e3e91ac86 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:31.333882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b4a19-9205-42d1-9b0c-4f0e3e91ac86 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:31.333980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b4a19-9205-42d1-9b0c-4f0e3e91ac86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:31.411982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-614b4a19-9205-42d1-9b0c-4f0e3e91ac86 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:31.413239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-614b4a19-9205-42d1-9b0c-4f0e3e91ac86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:31.440839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:31.513944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:31.664832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443434c-0634-4fc8-ae82-90e77187fb5d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:31.666336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4443434c-0634-4fc8-ae82-90e77187fb5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:31.818979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443434c-0634-4fc8-ae82-90e77187fb5d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:31.863895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:31.940436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:31.940972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:31.941288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:31.941569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:31.943436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:32.036214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:32.037886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66acc89-4fff-46ff-a2b7-3b9d891c681c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:32.055870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 713/1419] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:32.952747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189bb6-4f9e-497c-b28a-bcd40b8b89c3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.953357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189bb6-4f9e-497c-b28a-bcd40b8b89c3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.953484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a189bb6-4f9e-497c-b28a-bcd40b8b89c3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:32.955287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:32.960406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b4b77-9133-46e3-8092-22307a3d893c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:32.960868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b4b77-9133-46e3-8092-22307a3d893c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:32.960980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342b4b77-9133-46e3-8092-22307a3d893c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.021800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:33.045186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-342b4b77-9133-46e3-8092-22307a3d893c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:33.046394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-342b4b77-9133-46e3-8092-22307a3d893c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:33.147555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:33.305987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189bb6-4f9e-497c-b28a-bcd40b8b89c3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:33.307496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5a189bb6-4f9e-497c-b28a-bcd40b8b89c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:33.461428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a189bb6-4f9e-497c-b28a-bcd40b8b89c3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:33.474615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:33.586769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:33.587384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:33.587664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:33.587939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:33.589662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:33.687778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:33.689533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2dde3-c8e2-4777-b454-7ca5027c7876 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:33.709652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 708/1422] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:34.596926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b529b-ed94-46e8-8121-272001c0f8b1 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:34.597472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b529b-ed94-46e8-8121-272001c0f8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:34.597592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b529b-ed94-46e8-8121-272001c0f8b1 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:34.598884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:34.605204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868139f0-74d4-430a-94fd-57c73c5b8928 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:34.605669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868139f0-74d4-430a-94fd-57c73c5b8928 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:34.605764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868139f0-74d4-430a-94fd-57c73c5b8928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:34.631765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:34.700317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-868139f0-74d4-430a-94fd-57c73c5b8928 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:34.701570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-868139f0-74d4-430a-94fd-57c73c5b8928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:34.796191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:34.947669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b529b-ed94-46e8-8121-272001c0f8b1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:34.949285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-593b529b-ed94-46e8-8121-272001c0f8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:35.068927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:35.105522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593b529b-ed94-46e8-8121-272001c0f8b1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:35.341216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:35.341744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:35.342020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:35.342291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:35.344511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:35.423181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:35.436694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:35.438374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd66687d-1f3d-4451-8ef2-75f56a0f3ec5 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:35.457273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 716/1425] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:36.334707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42326-5519-4eb8-b191-96cc6793d217 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.335259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42326-5519-4eb8-b191-96cc6793d217 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.335259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb42326-5519-4eb8-b191-96cc6793d217 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.336602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:36.342563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe127a33-7ce0-43ad-aae4-bba67e904c5b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.343136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe127a33-7ce0-43ad-aae4-bba67e904c5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.343136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe127a33-7ce0-43ad-aae4-bba67e904c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:36.428862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fe127a33-7ce0-43ad-aae4-bba67e904c5b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:36.430113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fe127a33-7ce0-43ad-aae4-bba67e904c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:36.534091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:36.693134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb42326-5519-4eb8-b191-96cc6793d217 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:36.694955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cdb42326-5519-4eb8-b191-96cc6793d217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:36.794754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:36.867128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb42326-5519-4eb8-b191-96cc6793d217 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:36.997324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:36.997980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:36.998261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:36.998535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:37.000816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:37.054338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:37.097570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:37.099181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ae10bb-9e42-4266-ba37-9c91486db5a6 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:37.118512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 711/1428] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:38.004774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ee149-d791-421c-9830-c38fc9753040 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:38.005384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ee149-d791-421c-9830-c38fc9753040 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:38.005384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ee149-d791-421c-9830-c38fc9753040 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:38.006717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:38.011880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990119-8ca6-4683-b70e-ca09df81796c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:38.012313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990119-8ca6-4683-b70e-ca09df81796c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:38.012465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33990119-8ca6-4683-b70e-ca09df81796c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:38.091497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-33990119-8ca6-4683-b70e-ca09df81796c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:38.092706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-33990119-8ca6-4683-b70e-ca09df81796c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:38.190203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:38.345246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ee149-d791-421c-9830-c38fc9753040 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:38.346691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c32ee149-d791-421c-9830-c38fc9753040 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:38.396805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:38.497989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ee149-d791-421c-9830-c38fc9753040 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:38.611080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:38.611638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:38.611914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:38.612183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:38.613838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:38.621192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:38.709405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:38.710991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8754083-8964-48c6-946b-dc0c38b4ab1a service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:38.730048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 719/1431] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:39.607312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7bc4-30f8-4285-828a-9a272d97c5d6 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:39.607905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7bc4-30f8-4285-828a-9a272d97c5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:39.607905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381a7bc4-30f8-4285-828a-9a272d97c5d6 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:39.610117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:39.615759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6a91-e524-4bea-bc66-7b012996dcc8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:39.616312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6a91-e524-4bea-bc66-7b012996dcc8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:39.616312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ae6a91-e524-4bea-bc66-7b012996dcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:39.700785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-57ae6a91-e524-4bea-bc66-7b012996dcc8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:39.702049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-57ae6a91-e524-4bea-bc66-7b012996dcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:39.772045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:39.802794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:39.957064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7bc4-30f8-4285-828a-9a272d97c5d6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:39.958596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-381a7bc4-30f8-4285-828a-9a272d97c5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:39.976783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:40.112802 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381a7bc4-30f8-4285-828a-9a272d97c5d6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:40.202564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:40.230238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:40.230841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:40.231116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:40.231381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:40.232973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:40.324466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:40.326101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f895f-88fa-4001-aaaf-a6aed23b1e99 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:40.345718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 714/1434] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:41.271109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86a23-452b-49a4-ac5c-74c3ee271a3f None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:41.271770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86a23-452b-49a4-ac5c-74c3ee271a3f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:41.271770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff86a23-452b-49a4-ac5c-74c3ee271a3f None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.273080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:41.278210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbca486-79dd-4f43-9454-4998a4b67ca2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:41.278682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbca486-79dd-4f43-9454-4998a4b67ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:41.278785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbca486-79dd-4f43-9454-4998a4b67ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.356838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8cbca486-79dd-4f43-9454-4998a4b67ca2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:41.358106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8cbca486-79dd-4f43-9454-4998a4b67ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:41.401185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:41.458476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:41.613460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff86a23-452b-49a4-ac5c-74c3ee271a3f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:41.614996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8ff86a23-452b-49a4-ac5c-74c3ee271a3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:41.768893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff86a23-452b-49a4-ac5c-74c3ee271a3f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:41.823021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:41.881842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:41.882445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:41.882721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:41.882987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:41.884569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:41.975732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:41.977335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b3969e-aceb-45d9-8965-b1b76f23447a service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:41.996169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 722/1437] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:42.928980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e163e-7486-4ae7-9750-83642009540f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:42.929590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e163e-7486-4ae7-9750-83642009540f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:42.929590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5e163e-7486-4ae7-9750-83642009540f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:42.930991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:42.936585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791c42-a7fc-4df1-a23b-d7e0bfdfe402 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:42.937172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791c42-a7fc-4df1-a23b-d7e0bfdfe402 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:42.937791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20791c42-a7fc-4df1-a23b-d7e0bfdfe402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:43.019249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-20791c42-a7fc-4df1-a23b-d7e0bfdfe402 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:43.020538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-20791c42-a7fc-4df1-a23b-d7e0bfdfe402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:43.022671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:43.119996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:24:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:43.270936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e163e-7486-4ae7-9750-83642009540f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:43.272501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9b5e163e-7486-4ae7-9750-83642009540f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:43.425317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5e163e-7486-4ae7-9750-83642009540f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:43.454116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:43.542073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:43.542675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:43.542953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:43.543221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:43.544828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:43.637243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:43.638880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88419c1-9710-49b7-b35c-ce84a56400dd service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:43.659330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 717/1440] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:45.554208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432a14f-9312-47cb-8ca9-1a1857228261 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:45.554983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432a14f-9312-47cb-8ca9-1a1857228261 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:45.554983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1432a14f-9312-47cb-8ca9-1a1857228261 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:45.556121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:24:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:45.562131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a0895-4989-4154-88f0-87f2a7aa1ad3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:45.562582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a0895-4989-4154-88f0-87f2a7aa1ad3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:45.562696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a0895-4989-4154-88f0-87f2a7aa1ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:45.605695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:45.632448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6e0a0895-4989-4154-88f0-87f2a7aa1ad3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:45.633779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6e0a0895-4989-4154-88f0-87f2a7aa1ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:45.733737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:24:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:46.018361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432a14f-9312-47cb-8ca9-1a1857228261 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:46.019882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1432a14f-9312-47cb-8ca9-1a1857228261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:46.125795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1432a14f-9312-47cb-8ca9-1a1857228261 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:46.160774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:46.207226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:46.207826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:46.208102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:46.208368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:46.209981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:46.320710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:46.322101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba8dae8-a204-4e4a-9940-7b6cef599cc8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:46.342888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 725/1443] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:48.563048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cf49a-af2a-4e0f-aa20-de3ba19d4581 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:48.563582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cf49a-af2a-4e0f-aa20-de3ba19d4581 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:48.563582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12cf49a-af2a-4e0f-aa20-de3ba19d4581 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:48.565001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:24:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:48.572207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdb2a-86cf-4598-bec0-a68ba68aeee1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:48.572753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdb2a-86cf-4598-bec0-a68ba68aeee1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:48.572753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827bdb2a-86cf-4598-bec0-a68ba68aeee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:48.651485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-827bdb2a-86cf-4598-bec0-a68ba68aeee1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:48.652712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-827bdb2a-86cf-4598-bec0-a68ba68aeee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:48.667245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:48.757398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:49.049022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cf49a-af2a-4e0f-aa20-de3ba19d4581 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:49.050596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b12cf49a-af2a-4e0f-aa20-de3ba19d4581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:49.164579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12cf49a-af2a-4e0f-aa20-de3ba19d4581 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:49.177480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:49.247208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:49.247824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:49.248108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:49.248383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:49.250239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:49.368059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:49.369527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948217ca-977b-430c-b740-615cb8bac2b7 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:49.389491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 720/1446] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:51.813293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcca70-acbe-4446-b5d1-e10ee2c6753e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.813818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcca70-acbe-4446-b5d1-e10ee2c6753e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.813818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bcca70-acbe-4446-b5d1-e10ee2c6753e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.815111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:24:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:51.820465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a124-ae2e-464a-a259-33045169417d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:51.820917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a124-ae2e-464a-a259-33045169417d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:51.821055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e4a124-ae2e-464a-a259-33045169417d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:51.871336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:51.890991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-98e4a124-ae2e-464a-a259-33045169417d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:51.893608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-98e4a124-ae2e-464a-a259-33045169417d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:52.007239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:52.298883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcca70-acbe-4446-b5d1-e10ee2c6753e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:52.300615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76bcca70-acbe-4446-b5d1-e10ee2c6753e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:52.411808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bcca70-acbe-4446-b5d1-e10ee2c6753e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:52.454339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:52.490919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:52.491528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:52.491799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:52.492062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:52.493685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:52.607497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:52.609997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f417e452-f484-4fc0-ace8-342f37a2655d service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:52.629322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 728/1449] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:24:54.715452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f435bf-d751-43b0-b6e8-e0437fbbf213 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.715989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f435bf-d751-43b0-b6e8-e0437fbbf213 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.716085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f435bf-d751-43b0-b6e8-e0437fbbf213 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.717506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:24:54.723166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e0b8a-24be-4079-80bd-f85056aaf01b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:54.723622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e0b8a-24be-4079-80bd-f85056aaf01b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:54.723714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e0b8a-24be-4079-80bd-f85056aaf01b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:54.792145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6c5e0b8a-24be-4079-80bd-f85056aaf01b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:54.793385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c5e0b8a-24be-4079-80bd-f85056aaf01b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:54.815857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:54.894708 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:24:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:24:55.185174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f435bf-d751-43b0-b6e8-e0437fbbf213 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:55.186889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e4f435bf-d751-43b0-b6e8-e0437fbbf213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:55.292388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f435bf-d751-43b0-b6e8-e0437fbbf213 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:55.313569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:55.368633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:55.369261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:55.369538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:55.369805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:55.371714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:55.484232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:55.485718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485edf4c-fde3-411d-a103-0c445605dfae service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:55.505234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 723/1452] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:24:57.653279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd33a4-f4ba-491c-aa09-52883a4de115 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:57.654028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd33a4-f4ba-491c-aa09-52883a4de115 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:57.654028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afd33a4-f4ba-491c-aa09-52883a4de115 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:57.655334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:24:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:24:57.661827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2cd5-edd4-4fc2-a377-b645483e3346 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:57.666754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2cd5-edd4-4fc2-a377-b645483e3346 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:57.666754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491f2cd5-edd4-4fc2-a377-b645483e3346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:57.745360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:57.748370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-491f2cd5-edd4-4fc2-a377-b645483e3346 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:24:57.750257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-491f2cd5-edd4-4fc2-a377-b645483e3346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:57.849938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:24:58.146129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd33a4-f4ba-491c-aa09-52883a4de115 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:24:58.147653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8afd33a4-f4ba-491c-aa09-52883a4de115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:58.256896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afd33a4-f4ba-491c-aa09-52883a4de115 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:24:58.307671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:24:58.333477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:24:58.334082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:24:58.334366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:24:58.334637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:24:58.336271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:24:58.447623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:24:58.449249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce304881-55fd-4ddb-988e-8fc1ad5761ec service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:24:58.467108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 731/1455] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:24:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:00.471441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d50f4-3649-4b87-a816-81c710383dcf None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:00.472023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d50f4-3649-4b87-a816-81c710383dcf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:00.472023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b5d50f4-3649-4b87-a816-81c710383dcf None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:00.473394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:00.479037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cf20f-7107-4fd2-8bd8-68b539d1ac2d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:00.480658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cf20f-7107-4fd2-8bd8-68b539d1ac2d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:00.480658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65cf20f-7107-4fd2-8bd8-68b539d1ac2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:00.551274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f65cf20f-7107-4fd2-8bd8-68b539d1ac2d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:00.552508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f65cf20f-7107-4fd2-8bd8-68b539d1ac2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:00.583605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:00.653049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:00.949106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d50f4-3649-4b87-a816-81c710383dcf None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:00.950661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6b5d50f4-3649-4b87-a816-81c710383dcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:01.056164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b5d50f4-3649-4b87-a816-81c710383dcf None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:01.085538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:01.132390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:01.133064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:01.133352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:01.133627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:01.135306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:01.245929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:01.247357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb8d730-72a2-4cb2-b1a6-100262bf35d0 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:01.267538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 726/1458] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:03.279599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac091-30b0-47a5-a1ea-5d14d954e2c9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:03.281258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac091-30b0-47a5-a1ea-5d14d954e2c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:03.281542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88eac091-30b0-47a5-a1ea-5d14d954e2c9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:03.283004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:25:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:03.289142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c310d-3ab2-4da9-88b6-b823920a093f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:03.289641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c310d-3ab2-4da9-88b6-b823920a093f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:03.289761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c310d-3ab2-4da9-88b6-b823920a093f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:03.357943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ff3c310d-3ab2-4da9-88b6-b823920a093f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:03.359216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ff3c310d-3ab2-4da9-88b6-b823920a093f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:03.367596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:03.577998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:03.875306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eac091-30b0-47a5-a1ea-5d14d954e2c9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:03.876940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-88eac091-30b0-47a5-a1ea-5d14d954e2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:03.990143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88eac091-30b0-47a5-a1ea-5d14d954e2c9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:04.046652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:04.065056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:04.065673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:04.065952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:04.066229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:04.067851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:04.182772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:04.184120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3682870f-23b9-48d3-a458-7d841581228a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:04.203068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 734/1461] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:25:06.198780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbbd56-d3f3-477a-9215-c2232abcae1a None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.199323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbbd56-d3f3-477a-9215-c2232abcae1a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.199323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5dbbd56-d3f3-477a-9215-c2232abcae1a None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.200656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:25:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:06.206185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36915df0-ae60-43ab-b571-9dd834304ed7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.206607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36915df0-ae60-43ab-b571-9dd834304ed7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.206731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36915df0-ae60-43ab-b571-9dd834304ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.275033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-36915df0-ae60-43ab-b571-9dd834304ed7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:06.276246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-36915df0-ae60-43ab-b571-9dd834304ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.329410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:06.389294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:25:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:06.675136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbbd56-d3f3-477a-9215-c2232abcae1a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:06.676671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5dbbd56-d3f3-477a-9215-c2232abcae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.780333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5dbbd56-d3f3-477a-9215-c2232abcae1a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:06.819017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:06.857001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:06.857662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:06.858565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:06.859059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:06.860925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:06.974761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:06.976197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f5aecd-7323-4622-9eab-7a89852ebd0e service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:06.995581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 729/1464] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:08.996058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90210a8-5bcd-41ba-aea7-6a6471d0c6e7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:08.996590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90210a8-5bcd-41ba-aea7-6a6471d0c6e7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:08.996590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90210a8-5bcd-41ba-aea7-6a6471d0c6e7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:08.997999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:25:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:09.003981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60ace2-49d9-4a4a-82db-48ea2fc4ea0b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:09.004436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60ace2-49d9-4a4a-82db-48ea2fc4ea0b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:09.004537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60ace2-49d9-4a4a-82db-48ea2fc4ea0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:09.073368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7d60ace2-49d9-4a4a-82db-48ea2fc4ea0b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:09.074519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7d60ace2-49d9-4a4a-82db-48ea2fc4ea0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:09.093814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:09.175058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:25:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:09.462668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90210a8-5bcd-41ba-aea7-6a6471d0c6e7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:09.464246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e90210a8-5bcd-41ba-aea7-6a6471d0c6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:09.570781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90210a8-5bcd-41ba-aea7-6a6471d0c6e7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:09.639040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:09.645797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:09.646448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:09.651636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:09.651636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:09.651636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:09.759831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:09.761215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a726e5-48d2-42ee-a822-d05600389564 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:09.781435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 737/1467] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:11.760851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaa028-2e93-4556-af76-cfd648905c7d None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:11.761454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaa028-2e93-4556-af76-cfd648905c7d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:11.761454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eaa028-2e93-4556-af76-cfd648905c7d None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:11.762822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:25:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:11.769878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecb0f2-2fac-495b-b074-bc59e3f9cf77 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:11.770435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecb0f2-2fac-495b-b074-bc59e3f9cf77 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:11.770546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ecb0f2-2fac-495b-b074-bc59e3f9cf77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:11.846017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-17ecb0f2-2fac-495b-b074-bc59e3f9cf77 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:11.847186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-17ecb0f2-2fac-495b-b074-bc59e3f9cf77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:11.897038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:11.945844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:25:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:12.232127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaa028-2e93-4556-af76-cfd648905c7d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:12.233942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-99eaa028-2e93-4556-af76-cfd648905c7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:12.338517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eaa028-2e93-4556-af76-cfd648905c7d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:12.390983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:12.419642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:12.424387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:12.424387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:12.424387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:12.424387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:12.546390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:12.547764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d2d8b1-963f-409b-818c-ee4f9df3f044 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:12.568241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 732/1470] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:21.949982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:22.565771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=66 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:32.708134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba8a63-d125-471f-9deb-e01f03243b16 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.708712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba8a63-d125-471f-9deb-e01f03243b16 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.708712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ba8a63-d125-471f-9deb-e01f03243b16 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.710389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 739/1471] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:32.715928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72dfdb-22eb-4ba9-a430-d21c510ddaea None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:32.716394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72dfdb-22eb-4ba9-a430-d21c510ddaea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:32.716682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72dfdb-22eb-4ba9-a430-d21c510ddaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:32.785040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cd72dfdb-22eb-4ba9-a430-d21c510ddaea None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:32.786288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cd72dfdb-22eb-4ba9-a430-d21c510ddaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:32.884756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 733/1472] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:33.185529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0bc9a-beea-496c-a793-ca99ec668f5b None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.185806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0bc9a-beea-496c-a793-ca99ec668f5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.186072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f0bc9a-beea-496c-a793-ca99ec668f5b None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.187486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 740/1473] 10.222.0.73 () {54 vars in 967 bytes} [Tue Aug 23 22:25:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:33.192966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d32780-b96f-41b5-b343-10d6c68fe4e2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.193280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d32780-b96f-41b5-b343-10d6c68fe4e2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.193557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d32780-b96f-41b5-b343-10d6c68fe4e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.272055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c6d32780-b96f-41b5-b343-10d6c68fe4e2 None None] MFA Rules not processed for user `c498027e07ce4bc28bc0518f22047b16`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:33.273289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c6d32780-b96f-41b5-b343-10d6c68fe4e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.293508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:33.348618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 734/1474] 10.222.0.73 () {58 vars in 1064 bytes} [Tue Aug 23 22:25:33 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:33.356895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0bc9a-beea-496c-a793-ca99ec668f5b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:33.358546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5f0bc9a-beea-496c-a793-ca99ec668f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.464989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f0bc9a-beea-496c-a793-ca99ec668f5b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:33.540832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:33.541463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:33.541743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:33.542014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:33.543614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:33.599631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:33.656390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:33.657855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9717d164-4def-4d52-a4aa-73629ca87de0 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:33.677321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 741/1475] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:35.714565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b5b0-b7c3-43c1-befb-3e568edb57ca None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:35.715131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b5b0-b7c3-43c1-befb-3e568edb57ca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:35.715131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce1b5b0-b7c3-43c1-befb-3e568edb57ca None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:35.717119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 735/1476] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:35.722590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa4bd0-e431-402b-9a81-2596e5c40e59 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:35.723053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa4bd0-e431-402b-9a81-2596e5c40e59 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:35.723162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faa4bd0-e431-402b-9a81-2596e5c40e59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:35.792775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7faa4bd0-e431-402b-9a81-2596e5c40e59 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:35.794084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7faa4bd0-e431-402b-9a81-2596e5c40e59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:35.893798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 742/1477] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:36.162742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b5b0-b7c3-43c1-befb-3e568edb57ca None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:36.164193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7ce1b5b0-b7c3-43c1-befb-3e568edb57ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:36.292251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce1b5b0-b7c3-43c1-befb-3e568edb57ca None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:36.345034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:36.395076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:36.395714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:36.395997 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:36.396269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:36.396977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:36.398847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:25:36.398847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:25:36.398847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:25:36.400081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602c408-0ac2-43d8-9a20-3dfd646bf45e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:36.422138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 736/1478] 10.222.0.73 () {56 vars in 1108 bytes} [Tue Aug 23 22:25:36 2022] GET /identity/v3/projects => generated 2553 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:25:38.435925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a5b05-e818-45ea-b7c9-4ff2dbf99a34 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.436499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a5b05-e818-45ea-b7c9-4ff2dbf99a34 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.436499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457a5b05-e818-45ea-b7c9-4ff2dbf99a34 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.437906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 743/1479] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:38.443274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ffed3-d526-406f-8758-ba59e46d7a30 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:38.443785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ffed3-d526-406f-8758-ba59e46d7a30 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:38.443927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9ffed3-d526-406f-8758-ba59e46d7a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:38.518351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5b9ffed3-d526-406f-8758-ba59e46d7a30 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:38.519582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5b9ffed3-d526-406f-8758-ba59e46d7a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:38.619526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 737/1480] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:38.918407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a5b05-e818-45ea-b7c9-4ff2dbf99a34 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:38.919969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-457a5b05-e818-45ea-b7c9-4ff2dbf99a34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:38.929792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:39.071523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457a5b05-e818-45ea-b7c9-4ff2dbf99a34 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:39.155328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:39.190299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:39.190913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:39.191193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:39.191477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:39.193147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:39.283949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:39.285578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e69a5-c203-4476-8332-627f2a89f014 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:39.304238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 744/1481] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:41.436791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec7b9-a875-46ac-8559-7356d40bc573 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.437484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec7b9-a875-46ac-8559-7356d40bc573 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.437484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87ec7b9-a875-46ac-8559-7356d40bc573 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.438719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 738/1482] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:41.444120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045304e-3079-4e9b-b4f9-186b9f2f59fc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:41.444605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045304e-3079-4e9b-b4f9-186b9f2f59fc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:41.444743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d045304e-3079-4e9b-b4f9-186b9f2f59fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:41.515952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d045304e-3079-4e9b-b4f9-186b9f2f59fc None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:41.517293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d045304e-3079-4e9b-b4f9-186b9f2f59fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.568145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:41.620868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 745/1483] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:41.915738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec7b9-a875-46ac-8559-7356d40bc573 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:41.917360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f87ec7b9-a875-46ac-8559-7356d40bc573 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:41.944518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:42.023830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87ec7b9-a875-46ac-8559-7356d40bc573 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:42.098657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:42.099267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:42.099540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:42.099804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:42.101457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:42.177430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:42.212776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:42.214257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2c3fe0-e02b-4cf6-b42e-91af4439a807 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:42.234299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 739/1484] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:44.270872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ab40-6a28-4d48-acca-2b3601a2a49d None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.271478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ab40-6a28-4d48-acca-2b3601a2a49d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.271478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2338ab40-6a28-4d48-acca-2b3601a2a49d None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.272834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 746/1485] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:44.278398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8ab71-6005-4395-8a19-829bd66a2aec None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.278834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8ab71-6005-4395-8a19-829bd66a2aec None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.278945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e8ab71-6005-4395-8a19-829bd66a2aec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.345505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-41e8ab71-6005-4395-8a19-829bd66a2aec None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:44.346712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-41e8ab71-6005-4395-8a19-829bd66a2aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:44.446278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 740/1486] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:44.736471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ab40-6a28-4d48-acca-2b3601a2a49d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:44.738054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2338ab40-6a28-4d48-acca-2b3601a2a49d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:44.841709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2338ab40-6a28-4d48-acca-2b3601a2a49d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:44.915317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:44.917242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:44.917848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:44.920509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:44.920819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:44.922528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:45.037470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:45.039666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fae9b7-2b3f-438d-9a47-0c902d683a27 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:45.059548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 747/1487] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:47.141977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826575e-ea8c-418a-aefa-be804bf97198 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:47.142491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826575e-ea8c-418a-aefa-be804bf97198 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:47.142491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e826575e-ea8c-418a-aefa-be804bf97198 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:47.143936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 741/1488] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:47.149254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69513716-9a95-473f-9bd9-a34ae48a280b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:47.149706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69513716-9a95-473f-9bd9-a34ae48a280b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:47.149814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69513716-9a95-473f-9bd9-a34ae48a280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:47.215997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-69513716-9a95-473f-9bd9-a34ae48a280b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:47.217266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69513716-9a95-473f-9bd9-a34ae48a280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:47.273628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:47.322828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 748/1489] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:47.616435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826575e-ea8c-418a-aefa-be804bf97198 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:47.618283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e826575e-ea8c-418a-aefa-be804bf97198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:47.655620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:47.732650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e826575e-ea8c-418a-aefa-be804bf97198 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:47.807934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:47.808459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:47.809364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:47.809655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:47.811266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:47.893033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:47.920936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:47.922401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68982c0-3f34-488b-942b-eb80c9785d9d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:47.940521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 742/1490] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:25:50.567269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee49cd-8a92-4d97-ade2-130bffcd95dd None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:50.567836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee49cd-8a92-4d97-ade2-130bffcd95dd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:50.567836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee49cd-8a92-4d97-ade2-130bffcd95dd None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:50.569238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 749/1491] 10.222.0.73 () {54 vars in 927 bytes} [Tue Aug 23 22:25:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:25:50.575544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79219a-7870-42a0-8c14-b07806d2ff93 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:50.576049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79219a-7870-42a0-8c14-b07806d2ff93 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:50.576150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79219a-7870-42a0-8c14-b07806d2ff93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:50.644464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0b79219a-7870-42a0-8c14-b07806d2ff93 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:25:50.645706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0b79219a-7870-42a0-8c14-b07806d2ff93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:50.744155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 743/1492] 10.222.0.73 () {58 vars in 1024 bytes} [Tue Aug 23 22:25:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:25:51.049034 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee49cd-8a92-4d97-ade2-130bffcd95dd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:25:51.050629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-58ee49cd-8a92-4d97-ade2-130bffcd95dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:51.155798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee49cd-8a92-4d97-ade2-130bffcd95dd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:25:51.233324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:25:51.233911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:25:51.234190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:25:51.234453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:25:51.236058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:25:51.245407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:25:51.347607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:25:51.349051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb566010-c60a-4b34-957d-2d0795afcec6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:25:51.366626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 750/1493] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:25:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:26:00.742345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=82 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:26:01.364344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=50 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:20.503384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:20.503976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:20.503976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:20.580776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:20.582019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:20.678377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 744/1494] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:27:20.686705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:20.687226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:20.687330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:20.764406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:20.765647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:20.864262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 751/1495] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:27:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:20.869878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:20.871086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:20.883562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a75337-e5ba-434e-a99a-5eadf1205d54 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:20.946889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:20.979764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:20.980369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:20.980640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:20.980904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:20.981577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:27:20.981875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:20.982864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:20.982864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:27:20.982864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:27:20.983744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:20.995454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 745/1496] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:27:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:27:21.003787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:21.014322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc21fa8-9812-40df-89e7-223b792d493f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:21.076328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:21.109659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:21.110387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:21.110591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:21.110864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:21.111506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:21.113280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:21.113280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:27:21.113280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:27:21.121760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:21.405358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 752/1497] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:27:21 2022] POST /identity/v3/projects => generated 377 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:21.409581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:21.411100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:21.445504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7370e97d-c6dc-4783-8b09-ca9d81b1ea2c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:21.541062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:21.541751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:21.541932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:21.542221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:21.542816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:21.543817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:21.543817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:27:21.543817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:27:21.545052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:21.731202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 746/1498] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:27:21 2022] POST /identity/v3/users => generated 289 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:27:21.735323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:21.736830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:21.800637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d8ebc5-66be-46a3-ba17-56298141eebf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:21.885355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:21.896210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:21.896816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:21.897131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:21.897434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:21.898080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:21.899128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:21.899128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:27:21.899128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:27:21.900019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:21.912713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 753/1499] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:27:21.917798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:21.919505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:21.931592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0648bc74-5d26-4d77-8516-4e6eb2a17ac2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:21.956705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:22.029221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:22.029745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:22.030026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:22.030299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:22.030943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:22.031951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:22.031951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:27:22.031951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:27:22.032836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c48e285-5a39-4c29-b8ec-f2b645de1ab1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:22.046033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 747/1500] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:27:21 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:27:22.050355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:22.051918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:22.084744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa5a0b-bfa4-4152-8b20-fed1bbbcc90d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:22.183621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:22.184289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:22.184574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:22.184851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] PATH_INFO: `/v3/projects/655dbb4689ff40e1a5014514d78c5826/users/15b6c4a52c3b4a92bddf6f8106991c7f/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:22.207672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=655dbb4689ff40e1a5014514d78c5826, user_id=15b6c4a52c3b4a92bddf6f8106991c7f, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:22.208721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:22.208721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:27:22.208721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:27:22.210752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868fb447-c104-48c0-9d98-cf6be87902a5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:22.306378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 754/1501] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:27:22 2022] PUT /identity/v3/projects/655dbb4689ff40e1a5014514d78c5826/users/15b6c4a52c3b4a92bddf6f8106991c7f/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:22.318569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe00c5-c899-4c4c-b6f9-79a203c3c1b0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:22.318855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe00c5-c899-4c4c-b6f9-79a203c3c1b0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:22.319141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fe00c5-c899-4c4c-b6f9-79a203c3c1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:22.375585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-31fe00c5-c899-4c4c-b6f9-79a203c3c1b0 None None] MFA Rules not processed for user `15b6c4a52c3b4a92bddf6f8106991c7f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:22.376824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-31fe00c5-c899-4c4c-b6f9-79a203c3c1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:22.467269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 748/1502] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:27:22 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:22.481255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66814a58-9a45-41b3-8c99-c2088ac613e7 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:22.481545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66814a58-9a45-41b3-8c99-c2088ac613e7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:22.481826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66814a58-9a45-41b3-8c99-c2088ac613e7 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:22.483174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 755/1503] 10.222.0.73 () {54 vars in 964 bytes} [Tue Aug 23 22:27:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:27:22.488082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec943-fc86-4591-87d7-0b66386bcfb8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:22.488620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec943-fc86-4591-87d7-0b66386bcfb8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:22.488896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bec943-fc86-4591-87d7-0b66386bcfb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:22.520456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:22.566551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-61bec943-fc86-4591-87d7-0b66386bcfb8 None None] MFA Rules not processed for user `f17bae22fe2b4aa09619bdc0b3e0b4a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:22.567776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-61bec943-fc86-4591-87d7-0b66386bcfb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:22.645518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 749/1504] 10.222.0.73 () {58 vars in 1061 bytes} [Tue Aug 23 22:27:22 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:22.654201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66814a58-9a45-41b3-8c99-c2088ac613e7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:22.655796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-66814a58-9a45-41b3-8c99-c2088ac613e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:22.710486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:22.765481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66814a58-9a45-41b3-8c99-c2088ac613e7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:22.840364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:22.840890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:22.841209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:22.841484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:22.843078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:22.942070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:22.943541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:22.947655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:22.962541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 756/1505] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:22 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:23.009569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56368e4d-8e58-4781-98da-3c608ff7d7e2 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:23.009981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56368e4d-8e58-4781-98da-3c608ff7d7e2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:23.010113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56368e4d-8e58-4781-98da-3c608ff7d7e2 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:23.011396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 750/1506] 10.222.0.73 () {52 vars in 830 bytes} [Tue Aug 23 22:27:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:27:23.015002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:23.016199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:23.119150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2a1a1f-9114-4bcc-b103-527a708a89df service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:23.231660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:23.232282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:23.232555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:23.232821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] PATH_INFO: `/v3/users/15b6c4a52c3b4a92bddf6f8106991c7f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:23.245895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b6c4a52c3b4a92bddf6f8106991c7f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:23.246905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:23.246905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:27:23.246905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:27:23.248108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:23.258489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:23.486472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b6c4a52c3b4a92bddf6f8106991c7f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:23.487015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b6c4a52c3b4a92bddf6f8106991c7f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:23.487434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b6c4a52c3b4a92bddf6f8106991c7f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:23.504777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b6c4a52c3b4a92bddf6f8106991c7f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:23.527540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b6c4a52c3b4a92bddf6f8106991c7f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:23.585069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 757/1507] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:27:23 2022] DELETE /identity/v3/users/15b6c4a52c3b4a92bddf6f8106991c7f => generated 0 bytes in 571 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:23.589778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:23.590222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:23.590316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:23.650449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:27:23.651693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:23.734289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:23.747624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 751/1508] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:27:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:27:23.757116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:23.758388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:23.863064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61653596-ae59-4e6b-83b7-c6fa3bdac58c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:23.920856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:23.940631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:23.941245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:23.941526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:23.941796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:23.943393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:24.052796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:24.055092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16afaff-aa9e-4def-a2be-7d7792285118 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:24.074283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 758/1509] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:27:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:27:24.644728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:27:24.646035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:27:24.664023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71afd30d-3ed0-42a7-ab89-b8447ce7bc7a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:27:24.767257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:27:24.767889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:27:24.768164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:27:24.768433 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] PATH_INFO: `/v3/projects/655dbb4689ff40e1a5014514d78c5826` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:27:24.779527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=655dbb4689ff40e1a5014514d78c5826)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:27:24.780570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:27:24.780570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:27:24.780570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:27:24.781798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:27:25.070440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '655dbb4689ff40e1a5014514d78c5826', 'request_id': 'req-10f47cc4-5747-471a-bb7a-5823c8242d42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:25.078325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '655dbb4689ff40e1a5014514d78c5826', 'request_id': 'req-10f47cc4-5747-471a-bb7a-5823c8242d42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:25.092560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '655dbb4689ff40e1a5014514d78c5826', 'request_id': 'req-10f47cc4-5747-471a-bb7a-5823c8242d42'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:25.226303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] The token cache is being invalidate because project 655dbb4689ff40e1a5014514d78c5826 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:27:25.226639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:25.227106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10f47cc4-5747-471a-bb7a-5823c8242d42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:27:25.228409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 752/1510] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:27:24 2022] DELETE /identity/v3/projects/655dbb4689ff40e1a5014514d78c5826 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:27:34.129195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=62 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:27:35.223467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:28:58.352014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a012f34-f0b8-4c1c-8ad6-0fffb05b4484 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:58.352631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a012f34-f0b8-4c1c-8ad6-0fffb05b4484 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:58.352631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a012f34-f0b8-4c1c-8ad6-0fffb05b4484 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:58.353951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:28:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:28:58.359612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ace8e-c336-49f1-987f-e2d3e1f0b244 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:58.360082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ace8e-c336-49f1-987f-e2d3e1f0b244 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:58.360197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ace8e-c336-49f1-987f-e2d3e1f0b244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:58.443688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-491ace8e-c336-49f1-987f-e2d3e1f0b244 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:58.444916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-491ace8e-c336-49f1-987f-e2d3e1f0b244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:58.551261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:28:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:58.708315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a012f34-f0b8-4c1c-8ad6-0fffb05b4484 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:58.710628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0a012f34-f0b8-4c1c-8ad6-0fffb05b4484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:58.832348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a012f34-f0b8-4c1c-8ad6-0fffb05b4484 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:58.837810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc5144c8-7fde-4e95-97c6-a54f28fbfc1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:58.838959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5144c8-7fde-4e95-97c6-a54f28fbfc1b service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:58.839388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5144c8-7fde-4e95-97c6-a54f28fbfc1b service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:58.839719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5144c8-7fde-4e95-97c6-a54f28fbfc1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:58.849455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5144c8-7fde-4e95-97c6-a54f28fbfc1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:58.851046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc5144c8-7fde-4e95-97c6-a54f28fbfc1b service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:58.861905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 760/1513] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:28:59.737162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6864bc4-9cd6-4847-8769-cbc08342808f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:59.737752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6864bc4-9cd6-4847-8769-cbc08342808f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:59.737752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6864bc4-9cd6-4847-8769-cbc08342808f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:59.739220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:28:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:28:59.744057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd85a7b-3516-4002-a353-6077df6f0b2a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:59.744465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd85a7b-3516-4002-a353-6077df6f0b2a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:59.744606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd85a7b-3516-4002-a353-6077df6f0b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:59.766342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4cd85a7b-3516-4002-a353-6077df6f0b2a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:28:59.767549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4cd85a7b-3516-4002-a353-6077df6f0b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:28:59.775260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:28:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:28:59.952806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6864bc4-9cd6-4847-8769-cbc08342808f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:28:59.968755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6864bc4-9cd6-4847-8769-cbc08342808f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:28:59.974288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d52a30-b456-496e-86ad-1d72f13162a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:28:59.974895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52a30-b456-496e-86ad-1d72f13162a9 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:28:59.975172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52a30-b456-496e-86ad-1d72f13162a9 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:28:59.975439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d52a30-b456-496e-86ad-1d72f13162a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:28:59.985765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52a30-b456-496e-86ad-1d72f13162a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:28:59.987379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d52a30-b456-496e-86ad-1d72f13162a9 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:28:59.998241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 755/1516] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:28:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:00.631516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873c989-e38a-4030-9344-e942bee8f46a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:00.631819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873c989-e38a-4030-9344-e942bee8f46a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:00.632094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873c989-e38a-4030-9344-e942bee8f46a None None] PATH_INFO: `/v3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:00.633606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 762/1517] 10.222.0.64 () {54 vars in 925 bytes} [Tue Aug 23 22:29:00 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:29:00.639821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5a19-989c-4d09-945e-d33c39a67194 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:00.640211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5a19-989c-4d09-945e-d33c39a67194 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:00.640535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f5a19-989c-4d09-945e-d33c39a67194 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:00.707285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b07f5a19-989c-4d09-945e-d33c39a67194 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:00.708507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b07f5a19-989c-4d09-945e-d33c39a67194 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:00.726837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:00.788320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 756/1518] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Aug 23 22:29:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:29:09.761521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=56 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:10.275339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e3a7e-1ff1-4cef-a6a1-edcb1db476e4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:10.276217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e3a7e-1ff1-4cef-a6a1-edcb1db476e4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:10.276522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e3a7e-1ff1-4cef-a6a1-edcb1db476e4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:10.278022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 763/1519] 10.222.0.73 () {54 vars in 961 bytes} [Tue Aug 23 22:29:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:10.283208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc0c98-b686-4c38-8906-fcfb758485b3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:10.283717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc0c98-b686-4c38-8906-fcfb758485b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:10.283822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63dc0c98-b686-4c38-8906-fcfb758485b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:10.304419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-63dc0c98-b686-4c38-8906-fcfb758485b3 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:10.305654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-63dc0c98-b686-4c38-8906-fcfb758485b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:10.316231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 757/1520] 10.222.0.73 () {58 vars in 1058 bytes} [Tue Aug 23 22:29:10 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:10.325265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e3a7e-1ff1-4cef-a6a1-edcb1db476e4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:10.340204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e3a7e-1ff1-4cef-a6a1-edcb1db476e4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:10.345243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf22bbfb-7ae3-4e73-9080-db38328b784a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:10.345854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22bbfb-7ae3-4e73-9080-db38328b784a service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:10.346129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22bbfb-7ae3-4e73-9080-db38328b784a service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:10.346396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf22bbfb-7ae3-4e73-9080-db38328b784a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:10.357862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22bbfb-7ae3-4e73-9080-db38328b784a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:10.359947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf22bbfb-7ae3-4e73-9080-db38328b784a service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:10.372480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 764/1521] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:29:11.262412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f1b1-90b0-4587-93ee-655b5cc01ccb None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:11.262722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f1b1-90b0-4587-93ee-655b5cc01ccb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:11.262994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f809f1b1-90b0-4587-93ee-655b5cc01ccb None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:11.264374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:29:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:11.269590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692cdd-d8e1-4caa-9b8b-a79d92f8cb54 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:11.270109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692cdd-d8e1-4caa-9b8b-a79d92f8cb54 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:11.270207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db692cdd-d8e1-4caa-9b8b-a79d92f8cb54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:11.291297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-db692cdd-d8e1-4caa-9b8b-a79d92f8cb54 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:11.292548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db692cdd-d8e1-4caa-9b8b-a79d92f8cb54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:11.301374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:29:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:11.460285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809f1b1-90b0-4587-93ee-655b5cc01ccb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:11.475695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f809f1b1-90b0-4587-93ee-655b5cc01ccb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:11.482135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7220fc-e93c-4d70-8d42-2f8095e12350 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:11.485275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7220fc-e93c-4d70-8d42-2f8095e12350 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:11.485275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7220fc-e93c-4d70-8d42-2f8095e12350 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:11.485275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7220fc-e93c-4d70-8d42-2f8095e12350 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:11.494905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7220fc-e93c-4d70-8d42-2f8095e12350 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:11.496730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7220fc-e93c-4d70-8d42-2f8095e12350 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:11.507145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 759/1524] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:12.335159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e29e-13d5-4a5c-a90b-642fc5d71e2a None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:12.335738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e29e-13d5-4a5c-a90b-642fc5d71e2a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:12.335738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9461e29e-13d5-4a5c-a90b-642fc5d71e2a None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:12.337086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 23 22:29:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:12.342218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ad5e9-07ed-4449-a985-25648744b8c2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:12.342533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ad5e9-07ed-4449-a985-25648744b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:12.342811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358ad5e9-07ed-4449-a985-25648744b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:12.362965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-358ad5e9-07ed-4449-a985-25648744b8c2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:12.364176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-358ad5e9-07ed-4449-a985-25648744b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:12.372984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 23 22:29:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:12.529898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461e29e-13d5-4a5c-a90b-642fc5d71e2a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:12.547810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9461e29e-13d5-4a5c-a90b-642fc5d71e2a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:12.553339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd683b1-6580-4114-b27a-b1c89cc70b0b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:12.553942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd683b1-6580-4114-b27a-b1c89cc70b0b service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:12.554224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd683b1-6580-4114-b27a-b1c89cc70b0b service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:12.554497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd683b1-6580-4114-b27a-b1c89cc70b0b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:12.563731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd683b1-6580-4114-b27a-b1c89cc70b0b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:12.565335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd683b1-6580-4114-b27a-b1c89cc70b0b service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:12.574791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 767/1527] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:16.553517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ec77-6b18-4c84-81c1-48d801d3e672 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:16.554155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ec77-6b18-4c84-81c1-48d801d3e672 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:16.554155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d50ec77-6b18-4c84-81c1-48d801d3e672 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:16.555858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:29:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:16.562518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2664919-03e3-4bae-b96f-90744c7109b2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:16.562931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2664919-03e3-4bae-b96f-90744c7109b2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:16.563131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2664919-03e3-4bae-b96f-90744c7109b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:16.583868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b2664919-03e3-4bae-b96f-90744c7109b2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:16.585115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2664919-03e3-4bae-b96f-90744c7109b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:16.594078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:29:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:16.894425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ec77-6b18-4c84-81c1-48d801d3e672 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:16.896207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2d50ec77-6b18-4c84-81c1-48d801d3e672 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:16.980853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d50ec77-6b18-4c84-81c1-48d801d3e672 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:16.984844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148df8f0-b156-4219-a4bd-b1ef2ab5c7df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:16.985488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148df8f0-b156-4219-a4bd-b1ef2ab5c7df service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:16.985769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148df8f0-b156-4219-a4bd-b1ef2ab5c7df service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:16.986045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148df8f0-b156-4219-a4bd-b1ef2ab5c7df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:16.996857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148df8f0-b156-4219-a4bd-b1ef2ab5c7df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:16.998366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148df8f0-b156-4219-a4bd-b1ef2ab5c7df service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:17.008459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 762/1530] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:18.971234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5281d4-c859-43d0-83ae-2b5bd6fa5866 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:18.971802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5281d4-c859-43d0-83ae-2b5bd6fa5866 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:18.971802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5281d4-c859-43d0-83ae-2b5bd6fa5866 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:18.973199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:29:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:18.979373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d95a8-4ee4-425e-a536-a55b88074704 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:18.979816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d95a8-4ee4-425e-a536-a55b88074704 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:18.979933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d95a8-4ee4-425e-a536-a55b88074704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:19.000537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6a7d95a8-4ee4-425e-a536-a55b88074704 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:19.001807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6a7d95a8-4ee4-425e-a536-a55b88074704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:19.010670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:29:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:19.299554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5281d4-c859-43d0-83ae-2b5bd6fa5866 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:19.311566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5281d4-c859-43d0-83ae-2b5bd6fa5866 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:19.315188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd85543-9c50-44a7-a668-ccf6d30a9bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:19.315791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd85543-9c50-44a7-a668-ccf6d30a9bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:19.316181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd85543-9c50-44a7-a668-ccf6d30a9bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:19.316342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd85543-9c50-44a7-a668-ccf6d30a9bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:19.326767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd85543-9c50-44a7-a668-ccf6d30a9bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:19.328205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd85543-9c50-44a7-a668-ccf6d30a9bc8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:19.338873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 770/1533] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:22.874171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a2d99-89a7-4d04-9d69-726a8fce88b0 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:22.874738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a2d99-89a7-4d04-9d69-726a8fce88b0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:22.874839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339a2d99-89a7-4d04-9d69-726a8fce88b0 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:22.876203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:29:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:22.881817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c34c-e8db-4aec-9ddf-f6b188735edb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:22.882327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c34c-e8db-4aec-9ddf-f6b188735edb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:22.882457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c823c34c-e8db-4aec-9ddf-f6b188735edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:22.903693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c823c34c-e8db-4aec-9ddf-f6b188735edb None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:22.904887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c823c34c-e8db-4aec-9ddf-f6b188735edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:22.912870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:29:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:23.521691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a2d99-89a7-4d04-9d69-726a8fce88b0 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:23.523440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-339a2d99-89a7-4d04-9d69-726a8fce88b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:23.593597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:23.608226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339a2d99-89a7-4d04-9d69-726a8fce88b0 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:23.611955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d07f18-ba9f-4557-b1c5-4284b97b68d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:23.617674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07f18-ba9f-4557-b1c5-4284b97b68d8 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:23.617674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07f18-ba9f-4557-b1c5-4284b97b68d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:23.617674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d07f18-ba9f-4557-b1c5-4284b97b68d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:23.623927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d07f18-ba9f-4557-b1c5-4284b97b68d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:23.625710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d07f18-ba9f-4557-b1c5-4284b97b68d8 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:23.637646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 765/1536] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:25.697350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f5ec5-115e-4a95-9a62-c9c3e7fb3429 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:25.697930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f5ec5-115e-4a95-9a62-c9c3e7fb3429 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:25.698042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7f5ec5-115e-4a95-9a62-c9c3e7fb3429 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:25.699386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 23 22:29:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:29:25.705124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3e79-ea57-4b44-a069-dd50ea325a07 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:25.705612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3e79-ea57-4b44-a069-dd50ea325a07 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:25.705757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30b3e79-ea57-4b44-a069-dd50ea325a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:25.726082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b30b3e79-ea57-4b44-a069-dd50ea325a07 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:29:25.727300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b30b3e79-ea57-4b44-a069-dd50ea325a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:25.737612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 23 22:29:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:29:26.320441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f5ec5-115e-4a95-9a62-c9c3e7fb3429 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:29:26.323202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c7f5ec5-115e-4a95-9a62-c9c3e7fb3429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:29:26.337498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7f5ec5-115e-4a95-9a62-c9c3e7fb3429 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:29:26.341831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c34f642-5692-4b2f-a578-bf5fc3bc9a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:29:26.342587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34f642-5692-4b2f-a578-bf5fc3bc9a34 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:29:26.342924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34f642-5692-4b2f-a578-bf5fc3bc9a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:29:26.343246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c34f642-5692-4b2f-a578-bf5fc3bc9a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:29:26.354158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34f642-5692-4b2f-a578-bf5fc3bc9a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:29:26.355806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c34f642-5692-4b2f-a578-bf5fc3bc9a34 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:29:26.367451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 773/1539] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:29:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:29:35.721108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=11 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:29:36.362326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=33 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:53.423240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.423861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.423861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.455908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:53.457336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:53.475239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 767/1540] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:53.484432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.485055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.485055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.518256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:53.520496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:53.530060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 774/1541] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:53.535240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:53.552177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe6b6f4-4583-40c1-a334-19e9d0b7557a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:53.567664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:53.569111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.569480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.569758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.570434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:53.570738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:53.572369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a55d5c1-2677-4c2d-a919-4024d1bd183a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:53.594886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 768/1542] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:53.604473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:53.619873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca36d62d-ca55-4e5f-a1d0-802010cd313f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:53.627836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1ad952-0912-47e5-87b1-7b24eb5fb743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:53.627836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad952-0912-47e5-87b1-7b24eb5fb743 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.627836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad952-0912-47e5-87b1-7b24eb5fb743 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.627836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad952-0912-47e5-87b1-7b24eb5fb743 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.628333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ad952-0912-47e5-87b1-7b24eb5fb743 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:53.629123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1ad952-0912-47e5-87b1-7b24eb5fb743 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:53.757619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.758224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.758723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.788387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:53.790121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:53.803130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 775/1544] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/projects => generated 371 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:53.803365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 769/1544] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:30:53.815041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:53.815759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6a5e8-469c-4405-8d4c-81daf65b6a30 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.816385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6a5e8-469c-4405-8d4c-81daf65b6a30 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.817234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f6a5e8-469c-4405-8d4c-81daf65b6a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.834135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493ccb57-3fb3-480a-8e69-ec057e8d7429 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:53.851183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b6f6a5e8-469c-4405-8d4c-81daf65b6a30 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:53.857977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6f6a5e8-469c-4405-8d4c-81daf65b6a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:53.939190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:53.939821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.940100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.940371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.941042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:53.943321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:53.966447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 776/1545] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:30:53.968396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f5962-bbca-4259-b4cf-b3a43d1f9fb1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:53.968933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f5962-bbca-4259-b4cf-b3a43d1f9fb1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:53.969507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92f5962-bbca-4259-b4cf-b3a43d1f9fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:53.994386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a92f5962-bbca-4259-b4cf-b3a43d1f9fb1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:53.995741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a92f5962-bbca-4259-b4cf-b3a43d1f9fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.005049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 777/1546] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.012166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3be587-eb22-48ec-a86e-ec8e0d8db156 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.012166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3be587-eb22-48ec-a86e-ec8e0d8db156 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.012166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3be587-eb22-48ec-a86e-ec8e0d8db156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.032579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-da3be587-eb22-48ec-a86e-ec8e0d8db156 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.033841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-da3be587-eb22-48ec-a86e-ec8e0d8db156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.048846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 778/1547] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.050674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f679de2-957e-4c78-8f14-dbd09de00870 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.051075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f679de2-957e-4c78-8f14-dbd09de00870 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.051873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f679de2-957e-4c78-8f14-dbd09de00870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.082471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 770/1548] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:53 2022] POST /identity/v3/users => generated 286 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:54.082983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0f679de2-957e-4c78-8f14-dbd09de00870 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.083358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.084346 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0f679de2-957e-4c78-8f14-dbd09de00870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.093887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 779/1549] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.094127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7988e271-c2e7-4596-b15c-08d65815fb5e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.095029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f844b5-3869-4830-b848-a6d6b81abaa8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.095316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f844b5-3869-4830-b848-a6d6b81abaa8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.095583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f844b5-3869-4830-b848-a6d6b81abaa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.099224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.100105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.100484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.100841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.101651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.101950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.103551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224d3c51-859f-4109-8e20-cdd355703304 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.117596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a6f844b5-3869-4830-b848-a6d6b81abaa8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.118824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a6f844b5-3869-4830-b848-a6d6b81abaa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.119862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 771/1550] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.121065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1534e6-b606-47d4-b0c9-2d3c393599a2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.121315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1534e6-b606-47d4-b0c9-2d3c393599a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.121590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1534e6-b606-47d4-b0c9-2d3c393599a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.127406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 780/1551] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.128562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f89c-8cb4-4444-9257-521729de2720 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.128855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f89c-8cb4-4444-9257-521729de2720 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.129196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e9f89c-8cb4-4444-9257-521729de2720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.150992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-42e9f89c-8cb4-4444-9257-521729de2720 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.152260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42e9f89c-8cb4-4444-9257-521729de2720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.154622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ef1534e6-b606-47d4-b0c9-2d3c393599a2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.155839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ef1534e6-b606-47d4-b0c9-2d3c393599a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.161510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 781/1552] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.164277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.164644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.164996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.166011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 772/1553] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.167150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e904-a5ad-4b52-a4a7-587af93abc8b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.167855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e904-a5ad-4b52-a4a7-587af93abc8b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.167855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8862e904-a5ad-4b52-a4a7-587af93abc8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.189394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.190807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.191460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8862e904-a5ad-4b52-a4a7-587af93abc8b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.193984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8862e904-a5ad-4b52-a4a7-587af93abc8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.200254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 782/1554] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.201161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.206731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 773/1555] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.210127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.211074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.211074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.212063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d472c9f0-3c5e-4d46-9f73-677ada416ba2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.216942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08822069-26aa-4639-81c9-5cce2b2eef9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.217596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08822069-26aa-4639-81c9-5cce2b2eef9d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.217875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08822069-26aa-4639-81c9-5cce2b2eef9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.218157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08822069-26aa-4639-81c9-5cce2b2eef9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.219042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08822069-26aa-4639-81c9-5cce2b2eef9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.220474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08822069-26aa-4639-81c9-5cce2b2eef9d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.231960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 783/1556] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:54.233280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0561329-0835-4f45-887d-58a09f8a8e49 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.233567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0561329-0835-4f45-887d-58a09f8a8e49 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.233841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0561329-0835-4f45-887d-58a09f8a8e49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.241001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.242300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.250530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 774/1557] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.253793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.255269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b0561329-0835-4f45-887d-58a09f8a8e49 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.256562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b0561329-0835-4f45-887d-58a09f8a8e49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.266078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d1fa81-79ef-47ff-8a61-ad24e133020f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.266372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 784/1558] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.266969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.267254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.267530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.274795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.275422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.275709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.275978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.276617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.277677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:30:54.277677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:30:54.277677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:30:54.278876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.288183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.289477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.297507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 785/1559] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.298311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.308588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a26a1bc-28e7-4ef6-b575-32842c1f872d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.313462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.314070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.314351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.314622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.315271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.315584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.316632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:30:54.316632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:30:54.316632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:30:54.318102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.330041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 786/1560] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.330841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.341580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b754a760-a522-456f-a066-35ff9d1287c3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.346361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.346954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.347236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.347507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.348184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.348487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.350209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9074ae4d-93ef-4641-8166-bb6ddccb915a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.363310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 787/1561] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.364695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.364943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.365249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.385349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:54.386567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:54.393120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 775/1562] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/projects => generated 375 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:54.394641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 788/1563] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:54.397761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.397913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.407691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f7063a-b5bd-4ac5-b9d1-fd9f8d6429a4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.432112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:54.462985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c6c265-ca9b-47a9-98d6-249f3a43f2bc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.466298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.466906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.467183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.467454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.467911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.468093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.468392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.468638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.468920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.469220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.469859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.470017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.471444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.482296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 776/1564] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:54.482840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 789/1565] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.485343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.485919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.494998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:54.496144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ccb4ad-c947-4765-876b-bb1ad05ec5ea admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.498990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15693a-7bfc-4890-ad38-41a9034155c4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.501459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.502059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.502338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.509688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.509688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.509688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.510055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.510055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.510055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.510055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.510449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.511328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.511328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.511328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.521499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 777/1566] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.522361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.532506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 790/1567] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.532506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.532816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16dd16b-7109-47cb-b7f4-4894affd672a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.537870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.538476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.538777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.539083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.539720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.541658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.550085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-983c4480-918c-4f25-bd19-44bdb1ad37fd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.550085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.550085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.550085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.550490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.550490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.551276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.672837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 778/1568] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/projects => generated 377 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:54.673717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.697130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 791/1569] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:54.697130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.744202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2fca84-c61a-4a97-8ff0-ea57a4d14e64 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.768477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b1ad66-543c-41d0-bdd2-a1aa4cb7a082 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.773900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.774443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.774726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.775003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.775658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:30:54.775964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.777659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.791045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 792/1570] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:30:54.791677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.802207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa59cf9-adf4-4d51-a4a8-c2ce1983e86f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.807206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.811220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.811220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.811220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] PATH_INFO: `/v3/projects/a9c8ce990a2c44a5972e361b371f051b/users/d07617b262c94dfd82dae715819c7d92/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.814720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.815329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.815608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.815883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.816509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.819458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.825156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9c8ce990a2c44a5972e361b371f051b, user_id=d07617b262c94dfd82dae715819c7d92, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.827859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.927591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 779/1571] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:54.927591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:54.936292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256c99f6-1263-45af-bf66-7adbcf08c4e4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:54.941279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:54.942662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:54.943047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:54.943393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:54.944074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:54.946008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:54.956396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 793/1572] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:54 2022] PUT /identity/v3/projects/a9c8ce990a2c44a5972e361b371f051b/users/d07617b262c94dfd82dae715819c7d92/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:54.957403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.019494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:55.024878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7940f0ef-e116-4c7b-9861-b5a1dc2d9752 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.029637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.030266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.030612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.030884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.031543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.033475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.091720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 780/1573] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.093274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.140929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:55.154803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 794/1574] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:54 2022] POST /identity/v3/projects => generated 385 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.155742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.158474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79342ae5-5248-4097-af66-c299c8534870 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.217782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.218497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.218760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.219040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.219689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.221684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.224628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a85369-0ff5-435a-893e-a3ebcf1945d8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.230392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.231004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.231283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.231553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.232189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.233818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:30:55.233818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:30:55.233818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:30:55.235053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.370110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 781/1575] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/projects => generated 393 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.370851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.391306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 795/1576] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/users => generated 289 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.407920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.407920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282259e7-af64-44aa-8430-8ef7d59d778b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.438499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64112e-dc70-4c79-9fd1-5bf5fd1f530f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.443712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.445959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.445959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.445959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.445959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.447431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.468876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.469529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.469808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.470079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.470771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.472767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.592312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 782/1577] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/users => generated 289 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.609359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.609359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f69c6d-247b-42fb-b906-70c3ba3498c2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.609359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff87e3e-8f0b-4dd7-8e9e-5309f94ef6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.609359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff87e3e-8f0b-4dd7-8e9e-5309f94ef6f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.609960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff87e3e-8f0b-4dd7-8e9e-5309f94ef6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.609960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff87e3e-8f0b-4dd7-8e9e-5309f94ef6f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.610483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff87e3e-8f0b-4dd7-8e9e-5309f94ef6f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.612165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff87e3e-8f0b-4dd7-8e9e-5309f94ef6f1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.622844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 783/1578] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:55.623985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.624269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.624545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.652815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 796/1579] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/projects => generated 373 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.652815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.657763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] MFA Rules not processed for user `d07617b262c94dfd82dae715819c7d92`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:55.657763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:55.704060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:55.715211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:55.720059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac735ea-df96-41ab-8348-40df0647bb14 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.725700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.726315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.726594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.726738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 784/1580] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:55.726926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.727613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.727730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.729827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.738710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb9238-cd54-4b95-95c1-9e439bfcbf2a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.744188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.745064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.745749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.746076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.746704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.748711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.855330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 797/1581] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/users => generated 289 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.856190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.865994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ad6f14-247e-419b-b52e-a455ae4ddb84 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.870881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.871471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.871755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.872030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.872656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.874782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.884674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 785/1582] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:55.890629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.897222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8046de7d-3b1a-4886-9c50-8e66f5848533 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.901970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.902699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.902992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.903268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.904012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.905770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.916990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 786/1583] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:55.917810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.927791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbddfe94-13fc-4299-85f8-06ce5007b3c2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.932991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.933652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.934094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.934441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.935147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.936849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.948508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 787/1584] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:55.949356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.960926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1c6bab-9f46-43fb-a1c0-2e8c4d113997 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.965608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.966333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.966616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.966952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.967614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.969310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:55.980074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 788/1585] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:55.980871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:55.990911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433ee2c-d043-4b0f-b5ec-1ccad4def26c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:55.996162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:55.997118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:55.997118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:55.997384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:55.998011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:55.999985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.061995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 798/1586] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/users => generated 297 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:56.063172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.073984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b924f8-e3b4-420b-a595-2bdd9c8526f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.078710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.079310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.079612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.079893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.080515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.083140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.090274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 789/1587] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:55 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:56.097981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.101746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1500299c-911c-4fe1-9124-fb757fc232a7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.106714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.107314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.107590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.107862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.108500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.110173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.121318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 790/1588] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.122144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.132532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660c48bb-ec9f-42f6-ac06-82fe781242fc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.137721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.137721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.138092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.138202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.138819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.140389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.151731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 791/1589] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.152536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.162744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a8452f-bb0d-4e69-a395-cbdd3ff2b2f9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.167592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.168255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.168539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.168817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.169555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.171193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.181773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 792/1590] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.182561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.192537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bf1b66-706f-4fbf-bf3c-484b1afa8858 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.198799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.199402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.199682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.199886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 799/1591] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:56 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:56.200035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.200717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.200838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.202629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.218452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 793/1592] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.218452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.226632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba779a23-e20c-4c49-83b8-1f899e91625b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.269040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251b8f8a-eeb2-4e67-beb4-7ac8ee465d3c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.274119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.274908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.275189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.275461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] PATH_INFO: `/v3/projects/928b74ee78134ef2a30168fcbda51f37/users/ca63244d301a4f61a31590383ded12b3/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.286565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.287425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.287712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.287986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.288633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.290333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.291616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=928b74ee78134ef2a30168fcbda51f37, user_id=ca63244d301a4f61a31590383ded12b3, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.294401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.301501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 794/1593] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.302907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.312255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25f6f998-732e-4850-8959-39182bcfa776 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.317735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.318473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.318751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.319024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.319655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.321254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.331190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 795/1594] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.331981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.342080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d256b716-f0b3-45f9-860e-40204210f39e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.349505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.349505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.349505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.349505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.350000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.350628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.362094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 796/1595] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.362914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.373893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621c5f0e-9e14-4b20-b7e0-80653865fa76 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.378621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.379911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.380361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.385111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.385111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.385111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.385111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:56.388057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 800/1596] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:56 2022] PUT /identity/v3/projects/928b74ee78134ef2a30168fcbda51f37/users/ca63244d301a4f61a31590383ded12b3/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:56.388933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.403513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 797/1597] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.405341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.415061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c75282-7690-4007-be29-d39efae7a74b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.462215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d7f8e-411f-4ca7-85f8-befc61b6c470 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.467253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.468053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.468439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.468788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] PATH_INFO: `/v3/projects/0430b3e5fcd0415fb2c658443d8f6bf1/users/2ca578d6a1374212a4f1476335c641b3/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.481192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.488424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.488424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.488424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.488424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.488424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.491321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=0430b3e5fcd0415fb2c658443d8f6bf1, user_id=2ca578d6a1374212a4f1476335c641b3, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.494023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.609504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 798/1598] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:56 2022] POST /identity/v3/users => generated 292 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:56.610325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.621382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0ae9d3-2087-47d1-aa1f-88a501c15316 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.625948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.626588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.626867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.627139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] PATH_INFO: `/v3/projects/250c10ecd98848ebbb2561dfe9a5dafa/users/83356e7f648442b590d70dd8e736e3f1/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.642909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 801/1599] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:56 2022] PUT /identity/v3/projects/0430b3e5fcd0415fb2c658443d8f6bf1/users/2ca578d6a1374212a4f1476335c641b3/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:56.642909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.646963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=250c10ecd98848ebbb2561dfe9a5dafa, user_id=83356e7f648442b590d70dd8e736e3f1, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.648003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:30:56.648003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:30:56.648003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:30:56.649986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.654472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:56.709069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de246a93-ada2-4458-b514-5b518715b1db admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.713691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.714301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.714580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.714856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.715505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.717186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.727752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 802/1600] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.728627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.738227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44d65bc-100c-4c43-bb18-84c0b9a7daac admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.743783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d413f7d9-522b-45f7-8fd5-9d35cce0d847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.744373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413f7d9-522b-45f7-8fd5-9d35cce0d847 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.744657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413f7d9-522b-45f7-8fd5-9d35cce0d847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.744931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d413f7d9-522b-45f7-8fd5-9d35cce0d847 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.745603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413f7d9-522b-45f7-8fd5-9d35cce0d847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.747186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d413f7d9-522b-45f7-8fd5-9d35cce0d847 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.752423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 799/1601] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:56 2022] PUT /identity/v3/projects/250c10ecd98848ebbb2561dfe9a5dafa/users/83356e7f648442b590d70dd8e736e3f1/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:56.753726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.765742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 803/1602] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.767530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.767787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.768064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.798002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] MFA Rules not processed for user `ca63244d301a4f61a31590383ded12b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:56.799590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:56.819796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1af05f0-b164-4242-94de-03db980ba51f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.824533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.825233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.825516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.826021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] PATH_INFO: `/v3/projects/5882c9c6cf794b3ab17230304c169f07/users/1c7b02646d0d499b8928aed95f10e9ac/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.842242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5882c9c6cf794b3ab17230304c169f07, user_id=1c7b02646d0d499b8928aed95f10e9ac, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.844903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.871683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 804/1603] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:56 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:56.872688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.883291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865048e3-c33f-42c1-a98d-170e5338498a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.888878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.889648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.890000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.890280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] PATH_INFO: `/v3/projects/9246005ab3b64f3db9fdf644a0b8be02/users/ee9f51ba0353486b8c978aa3a84f01e2/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.902738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 800/1604] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:56 2022] PUT /identity/v3/projects/5882c9c6cf794b3ab17230304c169f07/users/1c7b02646d0d499b8928aed95f10e9ac/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:56.903995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:56.907495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9246005ab3b64f3db9fdf644a0b8be02, user_id=ee9f51ba0353486b8c978aa3a84f01e2, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.910853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.951705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:56.969042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c985bffb-9ab8-449d-bf3a-c2625ef14242 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:56.974777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71238a2e-186d-4770-a22a-552904568fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:56.975413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238a2e-186d-4770-a22a-552904568fdd admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.975701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238a2e-186d-4770-a22a-552904568fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.976051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71238a2e-186d-4770-a22a-552904568fdd admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.976735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238a2e-186d-4770-a22a-552904568fdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:56.978375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71238a2e-186d-4770-a22a-552904568fdd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:56.989351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 801/1605] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:56.992207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e89d5-9ae4-4883-b6f4-9e3a6c66587c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:56.992591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e89d5-9ae4-4883-b6f4-9e3a6c66587c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:56.992928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29e89d5-9ae4-4883-b6f4-9e3a6c66587c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:56.997996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 805/1606] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:56 2022] PUT /identity/v3/projects/9246005ab3b64f3db9fdf644a0b8be02/users/ee9f51ba0353486b8c978aa3a84f01e2/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:56.998886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.024418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e29e89d5-9ae4-4883-b6f4-9e3a6c66587c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.025729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e29e89d5-9ae4-4883-b6f4-9e3a6c66587c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.064675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b092bb0b-7891-4201-9b21-0531a1b6d155 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.069770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a91784-113b-4615-aed0-40388c239fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.070475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a91784-113b-4615-aed0-40388c239fe5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.070817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a91784-113b-4615-aed0-40388c239fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.071173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a91784-113b-4615-aed0-40388c239fe5 admin admin] PATH_INFO: `/v3/projects/f4f9539894f147e0912a72a1855d31bc/users/ebbf49534a8f41109f51637307270016/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.074318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:57.086942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a91784-113b-4615-aed0-40388c239fe5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4f9539894f147e0912a72a1855d31bc, user_id=ebbf49534a8f41109f51637307270016, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.093068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41a91784-113b-4615-aed0-40388c239fe5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.093197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 802/1607] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.093197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.093197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.093197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.122087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] MFA Rules not processed for user `83356e7f648442b590d70dd8e736e3f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.133323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.191767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 803/1608] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.191767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.202336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc0e8f7c-62b6-4e79-bd0c-fe3d48dae06c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.206837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5ba2eb-0e25-4bba-9c2a-398ea5bead7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.208000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ba2eb-0e25-4bba-9c2a-398ea5bead7d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.208333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ba2eb-0e25-4bba-9c2a-398ea5bead7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.208554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5ba2eb-0e25-4bba-9c2a-398ea5bead7d admin admin] PATH_INFO: `/v3/projects/7694b0bc65c44bb3b32016a22088b36b/users/70bb89837b504764b43568a9b8a1abe0/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.224055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ba2eb-0e25-4bba-9c2a-398ea5bead7d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7694b0bc65c44bb3b32016a22088b36b, user_id=70bb89837b504764b43568a9b8a1abe0, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.226744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5ba2eb-0e25-4bba-9c2a-398ea5bead7d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.270814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 806/1609] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:56 2022] PUT /identity/v3/projects/f4f9539894f147e0912a72a1855d31bc/users/ebbf49534a8f41109f51637307270016/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:57.272439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ac2e1-4541-4b1d-be11-4bf138033836 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.272439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ac2e1-4541-4b1d-be11-4bf138033836 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.272661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416ac2e1-4541-4b1d-be11-4bf138033836 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.273926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 807/1610] 10.222.0.73 () {54 vars in 962 bytes} [Tue Aug 23 22:30:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:30:57.275206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.275516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.275757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.295106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.296301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.353898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 808/1611] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.354776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.362798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 804/1612] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:57 2022] PUT /identity/v3/projects/7694b0bc65c44bb3b32016a22088b36b/users/70bb89837b504764b43568a9b8a1abe0/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:57.363830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d532a2-c254-40f8-9e3b-94e0400e8923 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.364080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db180855-5fdb-4501-a285-4afa44071344 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.367932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db180855-5fdb-4501-a285-4afa44071344 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.367932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db180855-5fdb-4501-a285-4afa44071344 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.393085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db180855-5fdb-4501-a285-4afa44071344 None None] MFA Rules not processed for user `ee9f51ba0353486b8c978aa3a84f01e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.394430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db180855-5fdb-4501-a285-4afa44071344 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.422052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.422763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.423059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.423352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] PATH_INFO: `/v3/projects/0e380cf24cd64f99b5b84815973cabcc/users/7aa236f8411d4502bb0825ed0b522bf5/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.438309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e380cf24cd64f99b5b84815973cabcc, user_id=7aa236f8411d4502bb0825ed0b522bf5, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.441029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.458838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:57.462571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 805/1613] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.463390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db180855-5fdb-4501-a285-4afa44071344 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.519587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db180855-5fdb-4501-a285-4afa44071344 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.524490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 809/1614] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:57 2022] PUT /identity/v3/projects/0e380cf24cd64f99b5b84815973cabcc/users/7aa236f8411d4502bb0825ed0b522bf5/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:57.525676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.578347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25156507-87f9-4a49-95e5-6c96e2af8632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.579077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25156507-87f9-4a49-95e5-6c96e2af8632 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.579409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25156507-87f9-4a49-95e5-6c96e2af8632 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.579812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25156507-87f9-4a49-95e5-6c96e2af8632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.592137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffae34d-f5a1-4d1b-a6a5-a7af37118246 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.597237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.597886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.598240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.598949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.599642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.601823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.617683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:57.647174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25156507-87f9-4a49-95e5-6c96e2af8632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.648665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25156507-87f9-4a49-95e5-6c96e2af8632 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.659062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 806/1615] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:57.660396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb251193-51d0-42ee-b2bc-2fb7ef44e647 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.660714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb251193-51d0-42ee-b2bc-2fb7ef44e647 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.661089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb251193-51d0-42ee-b2bc-2fb7ef44e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.688503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-eb251193-51d0-42ee-b2bc-2fb7ef44e647 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.689771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-eb251193-51d0-42ee-b2bc-2fb7ef44e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.698522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 807/1616] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.699921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b6bb-6ea1-4e22-a14f-c3edff8d0277 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.700206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b6bb-6ea1-4e22-a14f-c3edff8d0277 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.700479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab27b6bb-6ea1-4e22-a14f-c3edff8d0277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.725858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 810/1617] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/projects => generated 381 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:57.725858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.725858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.747165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ab27b6bb-6ea1-4e22-a14f-c3edff8d0277 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.748740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ab27b6bb-6ea1-4e22-a14f-c3edff8d0277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.781728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7129e36-06ad-4c34-a4a2-347368d568b6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.785793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2fa33-e0a8-41d3-8a3e-f8da28f7a65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.786405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2fa33-e0a8-41d3-8a3e-f8da28f7a65a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.786685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2fa33-e0a8-41d3-8a3e-f8da28f7a65a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.786959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2fa33-e0a8-41d3-8a3e-f8da28f7a65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.826850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 808/1618] 10.222.0.73 () {58 vars in 1059 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.828198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.828547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.828889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.859095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] MFA Rules not processed for user `70bb89837b504764b43568a9b8a1abe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:57.861131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:57.864218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2fa33-e0a8-41d3-8a3e-f8da28f7a65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.865801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2fa33-e0a8-41d3-8a3e-f8da28f7a65a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.876389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 811/1619] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:57.877787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc144c-cd59-4782-a5f1-39b1990002f9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.878575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc144c-cd59-4782-a5f1-39b1990002f9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.878575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dc144c-cd59-4782-a5f1-39b1990002f9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.880359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 812/1620] 10.222.0.73 () {54 vars in 962 bytes} [Tue Aug 23 22:30:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:30:57.881265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc144c-cd59-4782-a5f1-39b1990002f9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.895560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dc144c-cd59-4782-a5f1-39b1990002f9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.902587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.903218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.903517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.903772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.904469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.906114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.916634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 813/1621] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:57.924445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.927551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3c498-fef5-497d-88c7-5f9f805cb0d3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.931184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.931773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.932059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.932317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.938751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 809/1622] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:57.939644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.942336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.943524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.952933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc650ddd-31c8-4497-9a95-40f7070e33bc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.953662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 814/1623] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:57.954401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:57.958762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.959392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.959673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.959946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:57.960578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:57.962535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:57.966373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedaf8b3-b054-476d-b214-f387a4a6120d service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:57.972227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a9106c-ae20-4170-8cf8-9c8d1b27ec1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:57.972969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9106c-ae20-4170-8cf8-9c8d1b27ec1d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:57.973474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9106c-ae20-4170-8cf8-9c8d1b27ec1d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:57.974245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a9106c-ae20-4170-8cf8-9c8d1b27ec1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.034280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9106c-ae20-4170-8cf8-9c8d1b27ec1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.036217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a9106c-ae20-4170-8cf8-9c8d1b27ec1d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.047514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 815/1624] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:58.048951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.049273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.049548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.058817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:58.072058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:58.073630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:58.085662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 816/1625] 10.222.0.73 () {58 vars in 1059 bytes} [Tue Aug 23 22:30:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:58.086628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.104350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 810/1626] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:30:57 2022] POST /identity/v3/users => generated 297 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:30:58.105690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.117428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d15e67e-5963-420d-9dbf-0be26e0e7558 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.130010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.130808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.132546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.132843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] PATH_INFO: `/v3/projects/0e380cf24cd64f99b5b84815973cabcc/users/7aa236f8411d4502bb0825ed0b522bf5/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.134897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315ce201-1ab5-4f1d-99ed-e57b69a8b81a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.136827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e380cf24cd64f99b5b84815973cabcc, user_id=7aa236f8411d4502bb0825ed0b522bf5, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.139600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.151152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.153350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.153889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.154393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.159998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 817/1627] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:58 2022] PUT /identity/v3/projects/0e380cf24cd64f99b5b84815973cabcc/users/7aa236f8411d4502bb0825ed0b522bf5/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:58.161621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.236262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.238413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.240928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1f38e-c697-4631-8b7a-5fa77c3ca84a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.246891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258bde95-817d-4f91-8407-e6221c1ca05d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.248530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bde95-817d-4f91-8407-e6221c1ca05d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.248530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bde95-817d-4f91-8407-e6221c1ca05d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.248530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258bde95-817d-4f91-8407-e6221c1ca05d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.248827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bde95-817d-4f91-8407-e6221c1ca05d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.250754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258bde95-817d-4f91-8407-e6221c1ca05d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.265187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 818/1628] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:58.266196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.266537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.266816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.276365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 811/1629] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:58.278175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.297445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee3959-9a1f-46ea-a20f-f67a06bf5506 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.308906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.309174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] MFA Rules not processed for user `7aa236f8411d4502bb0825ed0b522bf5`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:58.309622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.309902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.310203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] PATH_INFO: `/v3/projects/c111e05d7a06499682d04040c7367d8b/users/a771f59cbd74436ea258781928f279d2/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.310324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:58.347520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c111e05d7a06499682d04040c7367d8b, user_id=a771f59cbd74436ea258781928f279d2, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.354482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.411200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:58.420117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 819/1630] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:58 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:58.424863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.475903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:58.482180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 812/1631] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:58 2022] PUT /identity/v3/projects/c111e05d7a06499682d04040c7367d8b/users/a771f59cbd74436ea258781928f279d2/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:58.483093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.554212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a84d0a-9e6c-41b8-9d78-90d3a59f7fa9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.580637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ea7e02-4837-46fc-9c01-65a69ea76686 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.585636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.586378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.586802 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.587328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.640653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14628a50-94d9-407f-9e7d-abea02816dce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.641405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14628a50-94d9-407f-9e7d-abea02816dce service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.641926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14628a50-94d9-407f-9e7d-abea02816dce service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.642050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14628a50-94d9-407f-9e7d-abea02816dce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.664890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.666581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.681339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 813/1632] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:58 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:58.690117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.706887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:58.717402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14628a50-94d9-407f-9e7d-abea02816dce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.719683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14628a50-94d9-407f-9e7d-abea02816dce service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.734315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 820/1633] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:58.750219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d181a0-9a12-4b1e-a12f-6d4530b180c9 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.750444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d181a0-9a12-4b1e-a12f-6d4530b180c9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.750685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d181a0-9a12-4b1e-a12f-6d4530b180c9 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.756021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 821/1634] 10.222.0.73 () {54 vars in 912 bytes} [Tue Aug 23 22:30:58 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:30:58.764431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdb28-35c8-4a2f-879a-d5d5c42c2c99 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.764832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdb28-35c8-4a2f-879a-d5d5c42c2c99 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.770354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1fdb28-35c8-4a2f-879a-d5d5c42c2c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.782255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54470f75-de45-4228-9d6c-19d70f2426da service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.788501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188eb47e-431f-4866-a9ab-f47ede4d4803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.789393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eb47e-431f-4866-a9ab-f47ede4d4803 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.789770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eb47e-431f-4866-a9ab-f47ede4d4803 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.790049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188eb47e-431f-4866-a9ab-f47ede4d4803 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.790712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188eb47e-431f-4866-a9ab-f47ede4d4803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.792348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188eb47e-431f-4866-a9ab-f47ede4d4803 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.807396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 814/1635] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:30:58.807396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834e6ed-13e6-439e-8358-77d47fac272e None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.807396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834e6ed-13e6-439e-8358-77d47fac272e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.807396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4834e6ed-13e6-439e-8358-77d47fac272e None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.808785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 815/1636] 10.222.0.73 () {54 vars in 912 bytes} [Tue Aug 23 22:30:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:30:58.808785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834e6ed-13e6-439e-8358-77d47fac272e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.814226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3d1fdb28-35c8-4a2f-879a-d5d5c42c2c99 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:58.815472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3d1fdb28-35c8-4a2f-879a-d5d5c42c2c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:58.839673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4834e6ed-13e6-439e-8358-77d47fac272e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.844866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 822/1637] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:30:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:58.853986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:58.854692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.854975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.855253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] PATH_INFO: `/v3/projects/c111e05d7a06499682d04040c7367d8b/users/a771f59cbd74436ea258781928f279d2/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.856056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674acb0-3d59-44ac-bb00-989c85217c5b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:58.856056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674acb0-3d59-44ac-bb00-989c85217c5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:58.856373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4674acb0-3d59-44ac-bb00-989c85217c5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:58.858862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c111e05d7a06499682d04040c7367d8b, user_id=a771f59cbd74436ea258781928f279d2, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:58.862887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:58.879872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 816/1638] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:30:58 2022] PUT /identity/v3/projects/c111e05d7a06499682d04040c7367d8b/users/a771f59cbd74436ea258781928f279d2/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:30:58.901274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:58.906091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4674acb0-3d59-44ac-bb00-989c85217c5b None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:58.906699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4674acb0-3d59-44ac-bb00-989c85217c5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:58.969308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:58.994660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca911ce-5935-477b-b566-d67e6393653e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:58.999321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:59.000072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:59.000511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:59.000623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:59.001164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 823/1639] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:30:58 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:30:59.010654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:59.010654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:59.010654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:59.014740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:59.016245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:59.032042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 817/1640] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:30:59.053532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] MFA Rules not processed for user `a771f59cbd74436ea258781928f279d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:30:59.055158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:30:59.055435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:30:59.067431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e375dc10-02be-4bb3-b66b-0a61ee2b1f90 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:30:59.080939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:30:59.081629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:30:59.081916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:30:59.082781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:30:59.100849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:30:59.105278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:30:59.107394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:30:59.132664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 818/1641] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:30:59 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:30:59.177667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 824/1642] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:30:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:00.787527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:00.797036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13b005e-534a-414f-bbbd-2fdb669e5faf service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:00.801742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:00.802475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:00.802752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:00.803028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:00.814310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:00.816120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:00.828946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 819/1643] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:00 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:01.304456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:01.328411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e4150a-29a0-4ca9-b7c8-f932ce1c2ee2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:01.333955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445df834-3038-4117-b92a-b6c0dacf6e67 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:01.334813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df834-3038-4117-b92a-b6c0dacf6e67 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:01.335227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df834-3038-4117-b92a-b6c0dacf6e67 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:01.335986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445df834-3038-4117-b92a-b6c0dacf6e67 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:01.476524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445df834-3038-4117-b92a-b6c0dacf6e67 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:01.476524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445df834-3038-4117-b92a-b6c0dacf6e67 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:01.493679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 825/1644] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:01.679570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:01.877599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df51307c-e774-4a9f-acc2-74288a98bbfe service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:01.877599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:01.877599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:01.877599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:01.878484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] PATH_INFO: `/v3/users/ca63244d301a4f61a31590383ded12b3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:01.878484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca63244d301a4f61a31590383ded12b3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:01.879555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: WARNING py.warnings [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:31:01.879555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: warnings.warn(msg) Aug 23 22:31:01.879555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: Aug 23 22:31:01.882154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:02.744966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca63244d301a4f61a31590383ded12b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:02.745730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca63244d301a4f61a31590383ded12b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:02.810180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca63244d301a4f61a31590383ded12b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:02.812883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca63244d301a4f61a31590383ded12b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:02.845165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca63244d301a4f61a31590383ded12b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:02.977581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 820/1645] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:01 2022] DELETE /identity/v3/users/ca63244d301a4f61a31590383ded12b3 => generated 0 bytes in 1299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:02.984508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:02.984508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:02.985296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:03.057877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:03.059300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:03.214736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 826/1646] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:03.232483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:03.235669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:03.303709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:03.304134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:03.327451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:03.334480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5953ea5-92d6-4bb1-bc7a-8927cf431bc4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:03.349561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:03.350130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:03.350413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:03.350691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] PATH_INFO: `/v3/users/d07617b262c94dfd82dae715819c7d92` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:03.352296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d07617b262c94dfd82dae715819c7d92)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:03.354742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:03.400111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8acd0485-c8e2-4a96-9341-678939587925 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:03.416815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe5d453-c4eb-4a3a-9a9f-11c75561c36c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:03.418440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5d453-c4eb-4a3a-9a9f-11c75561c36c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:03.419079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5d453-c4eb-4a3a-9a9f-11c75561c36c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:03.419748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe5d453-c4eb-4a3a-9a9f-11c75561c36c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:03.432767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5d453-c4eb-4a3a-9a9f-11c75561c36c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:03.435088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe5d453-c4eb-4a3a-9a9f-11c75561c36c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:03.449034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 821/1647] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:03.703608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd07617b262c94dfd82dae715819c7d92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:03.705145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd07617b262c94dfd82dae715819c7d92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:03.711187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd07617b262c94dfd82dae715819c7d92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:03.744415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd07617b262c94dfd82dae715819c7d92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:03.759027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:03.794097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd07617b262c94dfd82dae715819c7d92'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:03.903820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 827/1648] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:03 2022] DELETE /identity/v3/users/d07617b262c94dfd82dae715819c7d92 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:03.913120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:03.913436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:03.914541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:04.002545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:04.007505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:04.214767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 822/1649] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:04.227656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:04.229602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:04.355799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a0eed5-cd69-429b-8c01-c07e28bdd843 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:04.361691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a59c1c2-420a-4aa3-bb3f-6cc7f6d46837 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:04.362402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c1c2-420a-4aa3-bb3f-6cc7f6d46837 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:04.365789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c1c2-420a-4aa3-bb3f-6cc7f6d46837 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:04.366750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a59c1c2-420a-4aa3-bb3f-6cc7f6d46837 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:04.398246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59c1c2-420a-4aa3-bb3f-6cc7f6d46837 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:04.399706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a59c1c2-420a-4aa3-bb3f-6cc7f6d46837 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:04.425892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 828/1650] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:04.701407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:04.708470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:04.761416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29171952-f1a1-481d-9aa2-0de582fd37ab None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:04.793003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:04.796364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:04.796703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:04.796984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] PATH_INFO: `/v3/projects/928b74ee78134ef2a30168fcbda51f37` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:04.803072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] RBAC: Authorizing `identity:delete_project(project_id=928b74ee78134ef2a30168fcbda51f37)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:04.813275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:05.065107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '928b74ee78134ef2a30168fcbda51f37', 'request_id': 'req-23470c99-7fce-4c9f-b96b-80a3db76b525'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:05.097265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '928b74ee78134ef2a30168fcbda51f37', 'request_id': 'req-23470c99-7fce-4c9f-b96b-80a3db76b525'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:05.125428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '928b74ee78134ef2a30168fcbda51f37', 'request_id': 'req-23470c99-7fce-4c9f-b96b-80a3db76b525'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:05.256175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] The token cache is being invalidate because project 928b74ee78134ef2a30168fcbda51f37 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:05.260822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:05.260822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:05.260822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 823/1651] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:04 2022] DELETE /identity/v3/projects/928b74ee78134ef2a30168fcbda51f37 => generated 0 bytes in 559 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:05.264033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d90e99-a7ff-4331-98ab-487350b02912 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:05.264960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d90e99-a7ff-4331-98ab-487350b02912 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:05.265486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d90e99-a7ff-4331-98ab-487350b02912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:05.357795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-46d90e99-a7ff-4331-98ab-487350b02912 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:05.368557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-46d90e99-a7ff-4331-98ab-487350b02912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:05.713644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 829/1652] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 449 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:05.719075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:05.722896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348270c-cae0-44f5-a610-404815f45aea None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:05.724061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:05.727183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348270c-cae0-44f5-a610-404815f45aea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:05.731164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c348270c-cae0-44f5-a610-404815f45aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:05.758103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23470c99-7fce-4c9f-b96b-80a3db76b525 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:05.775017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:05.775744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:05.776370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:05.778146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] PATH_INFO: `/v3/users/70bb89837b504764b43568a9b8a1abe0` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:05.780795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70bb89837b504764b43568a9b8a1abe0)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:05.787726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:05.819418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c348270c-cae0-44f5-a610-404815f45aea None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:05.821578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c348270c-cae0-44f5-a610-404815f45aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:05.846110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 830/1653] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:05.852120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348270c-cae0-44f5-a610-404815f45aea None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:05.867435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c348270c-cae0-44f5-a610-404815f45aea None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:05.878526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:05.879753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:05.880036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:05.880467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:05.881340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:05.887348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:05.887348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:05.935565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 831/1654] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:31:05.938294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:05.961672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5520652-7d1e-457c-a294-9a3ecea26355 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:05.985749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:05.986840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:05.986840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:05.987078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:05.987535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:05.995815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:06.119984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70bb89837b504764b43568a9b8a1abe0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:06.126277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70bb89837b504764b43568a9b8a1abe0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:06.141852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70bb89837b504764b43568a9b8a1abe0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:06.143791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70bb89837b504764b43568a9b8a1abe0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:06.160801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-657970fd-6260-4b8e-a776-83347851a146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70bb89837b504764b43568a9b8a1abe0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:06.166080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 832/1655] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:05 2022] POST /identity/v3/projects => generated 405 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:06.166783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:06.174617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:06.265879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:06.312505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 824/1656] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:05 2022] DELETE /identity/v3/users/70bb89837b504764b43568a9b8a1abe0 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:06.320133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.320656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.321277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.346575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c89a6a-5318-4e6a-8a08-1d25898bef5a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:06.358165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:06.359674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:06.477967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 825/1657] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:06.502738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:06.505449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:06.515177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:06.516161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.516796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.517893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.521265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:06.542263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:06.592602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:06.667940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3a398d-eeab-4b57-985a-3d195fdd03b4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:06.681004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 833/1658] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:06 2022] POST /identity/v3/users => generated 303 bytes in 515 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:06.681288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:06.682133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.682422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.684308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.687376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:06.709314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e83e9-d438-437d-b986-cad5b16a9ec2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:06.714058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:06.715850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:06.725288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:06.726021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.726383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.726779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.729713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 826/1659] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:06.730542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:06.732290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:06.765601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 834/1660] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:06.766241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:06.786997 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ce3fc8-2c36-4c65-a218-591e426de092 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:06.802486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:06.802486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.802486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.802962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.803833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:06.808365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:06.828957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 827/1661] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:06.833508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:06.844741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceefee4f-2f3c-44df-9daf-78e10364f5dc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:06.850855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7dc8f2-8264-4539-94f0-b846e9f8fba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:06.852285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dc8f2-8264-4539-94f0-b846e9f8fba5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.852285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dc8f2-8264-4539-94f0-b846e9f8fba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.852285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7dc8f2-8264-4539-94f0-b846e9f8fba5 admin admin] PATH_INFO: `/v3/projects/16c56a99c1164f5c853884e74eb011a7/users/429b0bed5085462281a8202dcb799e5b/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.866792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:06.870295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:06.873067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dc8f2-8264-4539-94f0-b846e9f8fba5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16c56a99c1164f5c853884e74eb011a7, user_id=429b0bed5085462281a8202dcb799e5b, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:06.875336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7dc8f2-8264-4539-94f0-b846e9f8fba5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:06.889198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dcaf60-fc3b-45b7-b3c5-8f51bff18b18 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:06.895908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:06.899859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:06.899859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:06.899859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] PATH_INFO: `/v3/projects/a9c8ce990a2c44a5972e361b371f051b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:06.899859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9c8ce990a2c44a5972e361b371f051b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:06.904366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:07.034777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 835/1662] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:06 2022] PUT /identity/v3/projects/16c56a99c1164f5c853884e74eb011a7/users/429b0bed5085462281a8202dcb799e5b/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:07.046545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.047500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.049527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.109692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9c8ce990a2c44a5972e361b371f051b', 'request_id': 'req-3336f33e-9ebe-422d-b7b7-ef2f0520407c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.122787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:07.125769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.127494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9c8ce990a2c44a5972e361b371f051b', 'request_id': 'req-3336f33e-9ebe-422d-b7b7-ef2f0520407c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.158769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9c8ce990a2c44a5972e361b371f051b', 'request_id': 'req-3336f33e-9ebe-422d-b7b7-ef2f0520407c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.288628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] The token cache is being invalidate because project a9c8ce990a2c44a5972e361b371f051b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:07.289083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.296211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.303869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 828/1663] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:06 2022] DELETE /identity/v3/projects/a9c8ce990a2c44a5972e361b371f051b => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:07.311151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:07.318147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.323449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 836/1664] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 278 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:07.365445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3336f33e-9ebe-422d-b7b7-ef2f0520407c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:07.371108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:07.371976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.372403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.372824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] PATH_INFO: `/v3/users/7aa236f8411d4502bb0825ed0b522bf5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.374274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7aa236f8411d4502bb0825ed0b522bf5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:07.377699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:07.419971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:07.420298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:07.586084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f8cb3c-dddc-4712-8adf-eddfd3400c47 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:07.596590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:07.598389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:07.598389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:07.598389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:07.614339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:07.616257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:07.622283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7aa236f8411d4502bb0825ed0b522bf5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.623734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7aa236f8411d4502bb0825ed0b522bf5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.632630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 837/1665] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:07.651008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7aa236f8411d4502bb0825ed0b522bf5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.654023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7aa236f8411d4502bb0825ed0b522bf5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.673985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7aa236f8411d4502bb0825ed0b522bf5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:07.775748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 829/1666] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:07 2022] DELETE /identity/v3/users/7aa236f8411d4502bb0825ed0b522bf5 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:08.864935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:08.878078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-159f525e-116c-4f45-9a61-a150ae19f5d0 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.062683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.065321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.065608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.065885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] PATH_INFO: `/v3/projects/0e380cf24cd64f99b5b84815973cabcc` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.070392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e380cf24cd64f99b5b84815973cabcc)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.073229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: WARNING py.warnings [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 23 22:31:09.073229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: warnings.warn(msg) Aug 23 22:31:09.073229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: Aug 23 22:31:09.078713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.328888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.332084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:09.334912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e380cf24cd64f99b5b84815973cabcc', 'request_id': 'req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.357628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e380cf24cd64f99b5b84815973cabcc', 'request_id': 'req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.357917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f0fee-b803-4a9a-92f0-0a2829ef5271 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.471818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:09.500205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:09.501132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.501660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.502185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] PATH_INFO: `/v3/projects/7694b0bc65c44bb3b32016a22088b36b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.505076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7694b0bc65c44bb3b32016a22088b36b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:09.507615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:09.511420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e380cf24cd64f99b5b84815973cabcc', 'request_id': 'req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.580054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] The token cache is being invalidate because project 0e380cf24cd64f99b5b84815973cabcc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:09.580352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.585303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4aa4b4e9-4741-4468-a554-4a8f8c96ad3b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.591409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 838/1667] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:08 2022] DELETE /identity/v3/projects/0e380cf24cd64f99b5b84815973cabcc => generated 0 bytes in 723 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:09.596212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945ba9f-b041-48af-8552-1bfb41646541 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.606507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945ba9f-b041-48af-8552-1bfb41646541 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.606507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3945ba9f-b041-48af-8552-1bfb41646541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.661219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3945ba9f-b041-48af-8552-1bfb41646541 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:09.663973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3945ba9f-b041-48af-8552-1bfb41646541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:09.701133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7694b0bc65c44bb3b32016a22088b36b', 'request_id': 'req-10177ee7-2942-46cd-8371-bc36e0d31690'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.729528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:09.735123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7694b0bc65c44bb3b32016a22088b36b', 'request_id': 'req-10177ee7-2942-46cd-8371-bc36e0d31690'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.749666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7694b0bc65c44bb3b32016a22088b36b', 'request_id': 'req-10177ee7-2942-46cd-8371-bc36e0d31690'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.833614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 839/1668] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:09.838968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.838968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.838968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.888415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:09.891582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:09.892241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] The token cache is being invalidate because project 7694b0bc65c44bb3b32016a22088b36b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:09.892743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.894795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-10177ee7-2942-46cd-8371-bc36e0d31690 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:09.913210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 830/1669] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:09 2022] DELETE /identity/v3/projects/7694b0bc65c44bb3b32016a22088b36b => generated 0 bytes in 585 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:09.922407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984b9c-8008-435c-8864-f034fa0abea0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:09.922914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984b9c-8008-435c-8864-f034fa0abea0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:09.923639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed984b9c-8008-435c-8864-f034fa0abea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:09.952621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 840/1670] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:09.955475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:09.986928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ed984b9c-8008-435c-8864-f034fa0abea0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:09.987173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d5a81-be3c-4c73-94cd-f0c2e2075e58 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:09.991482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ed984b9c-8008-435c-8864-f034fa0abea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:10.002662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.004497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.007627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.008080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.009197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:10.012148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.014456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.031869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 831/1671] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:10.033713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.034025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.034306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.045688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 841/1672] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:10.054528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.077569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:10.078635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f539356-f93d-467c-a084-8cb51c8a8472 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.078842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:10.086397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.093207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.093207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.093207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.093207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.093207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.102943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 832/1673] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:10.111140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.136452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615890b3-728c-4eb4-8e2d-b88c5679237f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.161217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.161944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.164845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.165268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.166006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:10.174502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.179373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.197069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 833/1674] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:10.200731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.264809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1249af6-3cd4-4d63-b691-b7cad3370ce5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.273763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.274444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.274730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.275054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.275751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.276519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 842/1675] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:10 2022] POST /identity/v3/projects => generated 357 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:10.278092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.284212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.426374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 834/1676] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:10 2022] POST /identity/v3/projects => generated 365 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:10.426891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.549101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7e7479-3c6d-4bec-97cc-20c02ae6d3e6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.565789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ba2d74-33ba-47c0-bc1c-88bfa8f2f80a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.575198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.576741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.577406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.584985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.584985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.584985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.700903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.701959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.701959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.702174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.702717 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.704742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.737480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 835/1677] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:10 2022] POST /identity/v3/users => generated 283 bytes in 312 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:10.742678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.756246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:10.758953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed01810-71d2-4829-8490-13402ca856ec admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.773321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.773321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.773321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.773321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.804159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.811718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.836974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 836/1678] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:10.841587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.877669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66b7a3f-9791-4e32-91a7-59c022a27553 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.877802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 843/1679] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:10 2022] POST /identity/v3/users => generated 279 bytes in 594 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:10.883277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.891072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.891663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.891935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.893785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.897633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.900849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.903694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff157de-d388-4f9f-b648-15c7d3ea71db admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.923440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 837/1680] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:10.923523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.926274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.926671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.927016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:10.927901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:10.930324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.931322 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:10.963329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8dde708-1958-4843-971c-b3b01ce13bff admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:10.976572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 844/1681] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:10.977996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:10.981295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc09f109-0d06-4788-b903-efe232a23592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:10.982210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09f109-0d06-4788-b903-efe232a23592 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:10.984439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09f109-0d06-4788-b903-efe232a23592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:10.984880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc09f109-0d06-4788-b903-efe232a23592 admin admin] PATH_INFO: `/v3/projects/2d975a3d83944540a82e8114322c3d27/users/5e17763e0a85482aaf468a7c2c3e9776/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.025998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9aa164-c3f1-44d6-bc53-ca6fe125a06b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.032643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09f109-0d06-4788-b903-efe232a23592 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d975a3d83944540a82e8114322c3d27, user_id=5e17763e0a85482aaf468a7c2c3e9776, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.033173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.033850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.034129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.034397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.043116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc09f109-0d06-4788-b903-efe232a23592 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.043651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.057427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.076238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 845/1682] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:11.080450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:11.092982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be47b6f1-1ce1-4155-92f2-fc80ccb3c61e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.109461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b06b53-766d-40cb-aa08-28de8b2f467c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.110365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b06b53-766d-40cb-aa08-28de8b2f467c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.110365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b06b53-766d-40cb-aa08-28de8b2f467c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.110365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b06b53-766d-40cb-aa08-28de8b2f467c admin admin] PATH_INFO: `/v3/projects/297c354a7753488c924e2e030bbdf7d6/users/2f347b5fc9d24e6bb94d51ea1915a601/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.138065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b06b53-766d-40cb-aa08-28de8b2f467c admin admin] RBAC: Authorizing `identity:create_grant(project_id=297c354a7753488c924e2e030bbdf7d6, user_id=2f347b5fc9d24e6bb94d51ea1915a601, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.140862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b06b53-766d-40cb-aa08-28de8b2f467c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.165063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 838/1683] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:10 2022] PUT /identity/v3/projects/2d975a3d83944540a82e8114322c3d27/users/5e17763e0a85482aaf468a7c2c3e9776/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:11.175321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.176627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.176917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.270208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] MFA Rules not processed for user `5e17763e0a85482aaf468a7c2c3e9776`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:11.276825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.296365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 846/1684] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:11 2022] PUT /identity/v3/projects/297c354a7753488c924e2e030bbdf7d6/users/2f347b5fc9d24e6bb94d51ea1915a601/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:11.310958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.311452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.311868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.408860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] MFA Rules not processed for user `2f347b5fc9d24e6bb94d51ea1915a601`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:11.417467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.454673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 839/1685] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:11.479701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:11.484342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.571297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 847/1686] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:11.572900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:11.655481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a902c1c3-d24c-4442-9116-d7800d766615 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.774158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ea1bff-7468-43bd-abf0-813586f4e4c3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:11.790486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1092af6d-6649-47a7-a4cf-71f4c15edfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.791127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092af6d-6649-47a7-a4cf-71f4c15edfdb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.791410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092af6d-6649-47a7-a4cf-71f4c15edfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.791680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1092af6d-6649-47a7-a4cf-71f4c15edfdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.792352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092af6d-6649-47a7-a4cf-71f4c15edfdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.794740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1092af6d-6649-47a7-a4cf-71f4c15edfdb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:11.822778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:11.823634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.824131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.824523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.875009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:11.926221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 848/1687] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/projects => generated 357 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:11.928730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b23a7-2880-4790-81a9-8ae8edbc8268 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:11.931750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b23a7-2880-4790-81a9-8ae8edbc8268 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:11.932305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0b23a7-2880-4790-81a9-8ae8edbc8268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:11.983437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-aa0b23a7-2880-4790-81a9-8ae8edbc8268 None None] MFA Rules not processed for user `1c7b02646d0d499b8928aed95f10e9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:11.984822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:11.985358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aa0b23a7-2880-4790-81a9-8ae8edbc8268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:11.987541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.012216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 840/1688] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:11 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:12.014242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.026679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3596733-18d7-415f-9cf3-5b459865d121 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.141343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 849/1689] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:11 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:12.155237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.156013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.156436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.185188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.185188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.185188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.185188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.185634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.193076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.201274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] MFA Rules not processed for user `1c7b02646d0d499b8928aed95f10e9ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:12.203088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:12.211831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 850/1690] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:12.223943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.226654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:12.292363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 841/1691] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/users => generated 285 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:12.294956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.309621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252c6dc4-bf2f-4b98-84dc-f9bf5f851952 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.319595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.320443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.320537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.320765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.321794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.323426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.357378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 842/1692] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:12.358356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.368393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf980c-6e9c-4e4a-8314-932456734a31 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.377324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96750081-68ee-45ea-b0b1-ca1e3759b261 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.378175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96750081-68ee-45ea-b0b1-ca1e3759b261 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.379166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96750081-68ee-45ea-b0b1-ca1e3759b261 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.379759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96750081-68ee-45ea-b0b1-ca1e3759b261 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.382585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9afe8-8b4b-457c-88b5-71edaf764b31 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.396870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96750081-68ee-45ea-b0b1-ca1e3759b261 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.397410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.398107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.398390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.398703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] PATH_INFO: `/v3/projects/f0f3239778914b19a8af3a503cf86665/users/135e93b060094c44b6eacedb6dcfcbb5/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.402729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96750081-68ee-45ea-b0b1-ca1e3759b261 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.425396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 851/1693] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:12.432656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f3239778914b19a8af3a503cf86665, user_id=135e93b060094c44b6eacedb6dcfcbb5, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.435961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.455130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.455583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.455961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.521459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:12.542181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] MFA Rules not processed for user `2ca578d6a1374212a4f1476335c641b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:12.544167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:12.617910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 843/1694] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:12 2022] PUT /identity/v3/projects/f0f3239778914b19a8af3a503cf86665/users/135e93b060094c44b6eacedb6dcfcbb5/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:12.632118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.703965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 852/1695] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:12.714066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.715728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:12.744159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8413f1d1-05be-4e71-96be-8cdb52cd707f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.755859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba27ea9a-c160-463d-802b-0b682986b255 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.762122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.762955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.763241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.763522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.764741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.767019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.782945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 844/1696] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:12.788331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:12.802837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09581520-5e88-475e-bbb7-1825a87d29ee admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:12.813598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e87914-31e8-4000-b88c-44df7578885b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.814399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87914-31e8-4000-b88c-44df7578885b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.814757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87914-31e8-4000-b88c-44df7578885b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.816926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e87914-31e8-4000-b88c-44df7578885b admin admin] PATH_INFO: `/v3/projects/f0f3239778914b19a8af3a503cf86665/users/135e93b060094c44b6eacedb6dcfcbb5/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.818731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e87914-31e8-4000-b88c-44df7578885b admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0f3239778914b19a8af3a503cf86665, user_id=135e93b060094c44b6eacedb6dcfcbb5, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:12.821503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e87914-31e8-4000-b88c-44df7578885b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:12.872502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 845/1697] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:12 2022] PUT /identity/v3/projects/f0f3239778914b19a8af3a503cf86665/users/135e93b060094c44b6eacedb6dcfcbb5/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:12.884797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f99c3b-0e97-4e63-89d0-69944a206a76 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.888050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f99c3b-0e97-4e63-89d0-69944a206a76 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.888170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:12.888278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f99c3b-0e97-4e63-89d0-69944a206a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.888354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:12.888560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:12.888842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:12.921969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-04f99c3b-0e97-4e63-89d0-69944a206a76 None None] MFA Rules not processed for user `135e93b060094c44b6eacedb6dcfcbb5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:12.923670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-04f99c3b-0e97-4e63-89d0-69944a206a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.002606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.010537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.034560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 853/1698] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:12 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:13.106690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 846/1699] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:12 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:13.121791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.215191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522b1dcf-1cf1-4277-9aab-2ec2b739b1fe service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.283897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef26244-7a6d-4070-b7a6-908b5ef7f818 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.284188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef26244-7a6d-4070-b7a6-908b5ef7f818 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.285261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef26244-7a6d-4070-b7a6-908b5ef7f818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.342088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.342088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.342088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.342088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.372572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0ef26244-7a6d-4070-b7a6-908b5ef7f818 None None] MFA Rules not processed for user `429b0bed5085462281a8202dcb799e5b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:13.375501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.377252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.378532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0ef26244-7a6d-4070-b7a6-908b5ef7f818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.418422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 854/1700] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:13 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:13.600583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 847/1701] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:13 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 318 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:13.612240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:13.615589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:13.700536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:INSERT=6,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:13.786907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073d3b8b-654b-4739-bc78-ad570c580256 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:13.798452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:13.799091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.799372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.799649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.800327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:13.804314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:13.880036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8fb1-7ddd-4801-b180-700bd5d375c8 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.880473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8fb1-7ddd-4801-b180-700bd5d375c8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.881627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2b8fb1-7ddd-4801-b180-700bd5d375c8 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.883244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:13.884192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 848/1702] 10.222.0.73 () {54 vars in 912 bytes} [Tue Aug 23 22:31:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:13.892575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046819c-424b-4fb9-bed2-60bc3c1a10b8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:13.893305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046819c-424b-4fb9-bed2-60bc3c1a10b8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:13.893967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4046819c-424b-4fb9-bed2-60bc3c1a10b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:13.992888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 855/1703] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:13 2022] POST /identity/v3/projects => generated 405 bytes in 381 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:13.996818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.068872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4046819c-424b-4fb9-bed2-60bc3c1a10b8 None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:14.071445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4046819c-424b-4fb9-bed2-60bc3c1a10b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:14.155816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806978f4-c0e3-4dc2-b511-ca51d49bbae8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.162076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.162718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.163001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.163283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.163908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.168424 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.204408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 849/1704] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:31:13 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:14.204922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.205231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.205505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.245991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] MFA Rules not processed for user `ebbf49534a8f41109f51637307270016`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:14.247639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:14.330481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 856/1705] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:13 2022] POST /identity/v3/users => generated 309 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:14.331180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.334522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 850/1706] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:14.337855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.337855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:14.363798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f101aec-fc8f-4b41-bbd1-2b8abf8063d6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.368274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.369199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.369619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.370047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.381393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.383554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.395931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 851/1707] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:14 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:14.397037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.412877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ddd5bf-f19b-4fc9-a4f8-075abd671d48 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.418733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104133ff-3b5a-44c6-80a9-32c2c8b0b6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.419353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104133ff-3b5a-44c6-80a9-32c2c8b0b6eb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.419630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104133ff-3b5a-44c6-80a9-32c2c8b0b6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.419923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104133ff-3b5a-44c6-80a9-32c2c8b0b6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.420549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104133ff-3b5a-44c6-80a9-32c2c8b0b6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.422960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104133ff-3b5a-44c6-80a9-32c2c8b0b6eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.436274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 852/1708] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:14.443395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.443395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.443395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.463547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3641af-29a0-4d4e-a2b8-aea1f21b758d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.467702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] MFA Rules not processed for user `ebbf49534a8f41109f51637307270016`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:14.469237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:14.473549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.475498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.475498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.475498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.481073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 853/1709] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:14 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:14.481073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.535790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba281c6c-4426-4be3-a68d-dd614263c49c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.540631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.541480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.541858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.542244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] PATH_INFO: `/v3/projects/3a12407f664448bc864f6e8638f1a107/users/8cb8aa5ce99d4ab98fed3e5b70fff4b7/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.589046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a12407f664448bc864f6e8638f1a107, user_id=8cb8aa5ce99d4ab98fed3e5b70fff4b7, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.592167 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.650902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.653962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.673521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 857/1710] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:14.674448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.727203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 854/1711] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:14 2022] PUT /identity/v3/projects/3a12407f664448bc864f6e8638f1a107/users/8cb8aa5ce99d4ab98fed3e5b70fff4b7/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:14.731247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.786123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9c4167-cb83-43cf-b4bc-1b19a0497854 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.826498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b502195-a9a7-4abd-af9d-39a7cbf58ce5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.832330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.833102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.833488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.833868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.834676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.836624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.848829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 855/1712] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:14.853225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:14.869077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1effe448-5ed0-4747-ac80-5d73b4eb532d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:14.874523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026fe02a-cd58-4023-baab-2f11ee102129 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.875385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe02a-cd58-4023-baab-2f11ee102129 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.875671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe02a-cd58-4023-baab-2f11ee102129 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.875791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97cd5c5f-50cf-4a0f-9881-543816604d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:14.875994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026fe02a-cd58-4023-baab-2f11ee102129 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.879668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd5c5f-50cf-4a0f-9881-543816604d3a admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:14.879668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd5c5f-50cf-4a0f-9881-543816604d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:14.879668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97cd5c5f-50cf-4a0f-9881-543816604d3a admin admin] PATH_INFO: `/v3/projects/3a12407f664448bc864f6e8638f1a107/users/8cb8aa5ce99d4ab98fed3e5b70fff4b7/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:14.881241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd5c5f-50cf-4a0f-9881-543816604d3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a12407f664448bc864f6e8638f1a107, user_id=8cb8aa5ce99d4ab98fed3e5b70fff4b7, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.884384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97cd5c5f-50cf-4a0f-9881-543816604d3a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:14.901332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 856/1713] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:14 2022] PUT /identity/v3/projects/3a12407f664448bc864f6e8638f1a107/users/8cb8aa5ce99d4ab98fed3e5b70fff4b7/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:14.995872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe02a-cd58-4023-baab-2f11ee102129 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:14.995872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-026fe02a-cd58-4023-baab-2f11ee102129 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:15.013925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 858/1714] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:14 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:15.500555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d3ff-65cd-41f9-811b-09400c7bed6c None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:15.501840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d3ff-65cd-41f9-811b-09400c7bed6c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:15.502408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2443d3ff-65cd-41f9-811b-09400c7bed6c None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:15.505485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 857/1715] 10.222.0.73 () {54 vars in 961 bytes} [Tue Aug 23 22:31:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:15.511334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e1503-f27a-4e78-bb46-86994ca78c49 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:15.512033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e1503-f27a-4e78-bb46-86994ca78c49 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:15.512033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13e1503-f27a-4e78-bb46-86994ca78c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:15.572870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c13e1503-f27a-4e78-bb46-86994ca78c49 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:15.576144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c13e1503-f27a-4e78-bb46-86994ca78c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:15.776589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 859/1716] 10.222.0.73 () {58 vars in 1058 bytes} [Tue Aug 23 22:31:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:15.780880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443d3ff-65cd-41f9-811b-09400c7bed6c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:15.834259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2443d3ff-65cd-41f9-811b-09400c7bed6c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:15.858769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f49d92-4ab2-4282-97bc-1084566917d4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:15.858769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f49d92-4ab2-4282-97bc-1084566917d4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:15.858769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f49d92-4ab2-4282-97bc-1084566917d4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:15.858769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 860/1717] 10.222.0.73 () {54 vars in 912 bytes} [Tue Aug 23 22:31:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 23 22:31:15.862057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:15.862222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc111cb-727f-4b2d-92af-3c896c296fd3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:15.862997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc111cb-727f-4b2d-92af-3c896c296fd3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:15.863377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc111cb-727f-4b2d-92af-3c896c296fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:15.864652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:15.865356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:15.865817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:15.867987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:15.885636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:15.890630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:15.905573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 858/1718] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:15.932578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5bc111cb-727f-4b2d-92af-3c896c296fd3 None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:15.933885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5bc111cb-727f-4b2d-92af-3c896c296fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:16.005605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:16.100909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 861/1719] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:31:15 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:16.111303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:16.113431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:16.133954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c52b2e-116a-43e7-b8c1-3d3330a19bdd service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:16.152419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1457cc95-e597-43b4-9c69-5623ce2e1875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:16.152419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457cc95-e597-43b4-9c69-5623ce2e1875 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:16.152419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457cc95-e597-43b4-9c69-5623ce2e1875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:16.152419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1457cc95-e597-43b4-9c69-5623ce2e1875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:16.176910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457cc95-e597-43b4-9c69-5623ce2e1875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:16.177324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1457cc95-e597-43b4-9c69-5623ce2e1875 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:16.244267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 859/1720] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:16 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:17.172975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:17.298888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5c0ae-82a5-48a6-a924-aa7aa848e081 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:17.300972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5c0ae-82a5-48a6-a924-aa7aa848e081 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:17.301427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb5c0ae-82a5-48a6-a924-aa7aa848e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:17.348816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2fb5c0ae-82a5-48a6-a924-aa7aa848e081 None None] MFA Rules not processed for user `ebbf49534a8f41109f51637307270016`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:17.352990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2fb5c0ae-82a5-48a6-a924-aa7aa848e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:17.519987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 862/1721] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:18.333099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:18.672865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:18.673393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:18.673481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:18.696893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] MFA Rules not processed for user `2ca578d6a1374212a4f1476335c641b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:18.698202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:18.718732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e961e4d-0b0e-4c25-b06d-ad9f195ec8fe None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:18.719062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e961e4d-0b0e-4c25-b06d-ad9f195ec8fe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:18.719336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e961e4d-0b0e-4c25-b06d-ad9f195ec8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:18.761533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4e961e4d-0b0e-4c25-b06d-ad9f195ec8fe None None] MFA Rules not processed for user `8cb8aa5ce99d4ab98fed3e5b70fff4b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:18.764931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4e961e4d-0b0e-4c25-b06d-ad9f195ec8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:18.782358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 860/1722] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:18.974347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 863/1723] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:18 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:19.351508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:20.739711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:20.741862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:20.978065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d041bd6-7352-463a-9200-903a463ae61d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:20.991892 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:20.994097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:21.001357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:21.001357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] PATH_INFO: `/v3/users/ee9f51ba0353486b8c978aa3a84f01e2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:21.026689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee9f51ba0353486b8c978aa3a84f01e2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:21.030920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:21.206722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55346cd6-5cca-4f5f-a3a7-4c7d24fdd6c3 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:21.207276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55346cd6-5cca-4f5f-a3a7-4c7d24fdd6c3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:21.207276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55346cd6-5cca-4f5f-a3a7-4c7d24fdd6c3 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:21.209273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 864/1724] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:31:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:21.218568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:21.218957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:21.219262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:21.301574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:21.303141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:21.316782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 865/1725] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:31:21 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:21.371187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee9f51ba0353486b8c978aa3a84f01e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:21.380706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee9f51ba0353486b8c978aa3a84f01e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:21.404036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee9f51ba0353486b8c978aa3a84f01e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:21.404818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee9f51ba0353486b8c978aa3a84f01e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:21.417522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee9f51ba0353486b8c978aa3a84f01e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:21.423928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:21.442171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748926c2-ce9d-4825-9ee0-090d7d46d8ac None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:21.450101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7086790d-421f-4102-a312-c29d83554690 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:21.451147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086790d-421f-4102-a312-c29d83554690 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:21.451636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086790d-421f-4102-a312-c29d83554690 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:21.452071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7086790d-421f-4102-a312-c29d83554690 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:21.455595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7086790d-421f-4102-a312-c29d83554690 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:21.474707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086790d-421f-4102-a312-c29d83554690 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:21.480218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7086790d-421f-4102-a312-c29d83554690 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:21.498808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 866/1726] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:21 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:21.549224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 861/1727] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:20 2022] DELETE /identity/v3/users/ee9f51ba0353486b8c978aa3a84f01e2 => generated 0 bytes in 807 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:21.560227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53c6d-8abb-47cb-894b-5c2c7b045c2c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:21.560576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53c6d-8abb-47cb-894b-5c2c7b045c2c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:21.561479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f53c6d-8abb-47cb-894b-5c2c7b045c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:21.670214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b6f53c6d-8abb-47cb-894b-5c2c7b045c2c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:21.670214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b6f53c6d-8abb-47cb-894b-5c2c7b045c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:21.788948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 867/1728] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:21.814336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:21.814336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:21.913204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252257ac-113a-49e9-86b9-9bff2a544997 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:21.917931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:21.923267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:21.923267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:21.923267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:21.932938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:21.934760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:21.952488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 862/1729] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:22.301596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:22.365855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:22.366145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:22.366603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:22.479888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:22.479888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:22.611896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 868/1730] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:31:22 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:22.737839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:22.739454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:22.771213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589d1dcb-522c-4d0e-941b-3c671298a2f2 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:22.806893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:22.808455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:22.808740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:22.809033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] PATH_INFO: `/v3/users/429b0bed5085462281a8202dcb799e5b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:22.812700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] RBAC: Authorizing `identity:delete_user(user_id=429b0bed5085462281a8202dcb799e5b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:22.819698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:23.117563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:23.119538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:23.259994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '429b0bed5085462281a8202dcb799e5b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:23.263432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '429b0bed5085462281a8202dcb799e5b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:23.302751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '429b0bed5085462281a8202dcb799e5b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:23.309761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '429b0bed5085462281a8202dcb799e5b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:23.340025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '429b0bed5085462281a8202dcb799e5b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:23.425589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:23.465105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d005c0-6294-4622-9410-c35c3e6b4cd5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:23.470829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:23.471537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:23.471949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:23.472232 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:23.474318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:23.517621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 863/1731] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:22 2022] DELETE /identity/v3/users/429b0bed5085462281a8202dcb799e5b => generated 0 bytes in 780 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:23.518702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:23.522777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:23.529956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:23.530934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaede98-e72a-4451-b913-49e63129df23 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:23.564294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 869/1732] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:31:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:31:23.568910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:23.572420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:23.572420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:23.662569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:23.675268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:23.911858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336ae9ba-8593-4911-8fe2-9d1afa5d4828 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:23.918980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:23.920792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:23.921769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:23.922390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] PATH_INFO: `/v3/projects/9246005ab3b64f3db9fdf644a0b8be02` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:23.925460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9246005ab3b64f3db9fdf644a0b8be02)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:23.928514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:23.993065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 870/1733] 10.222.0.64 () {58 vars in 1015 bytes} [Tue Aug 23 22:31:23 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 426 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:24.119576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:24.122422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:24.142840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bb56a8-c907-4dd1-84c8-79ce39090258 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:24.149113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6ec4c5-63b8-42b0-be2b-917dd8a5ebfb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:24.150556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ec4c5-63b8-42b0-be2b-917dd8a5ebfb service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.150556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ec4c5-63b8-42b0-be2b-917dd8a5ebfb service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.150556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6ec4c5-63b8-42b0-be2b-917dd8a5ebfb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:24.183039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ec4c5-63b8-42b0-be2b-917dd8a5ebfb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:24.188037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6ec4c5-63b8-42b0-be2b-917dd8a5ebfb service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:24.200598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9246005ab3b64f3db9fdf644a0b8be02', 'request_id': 'req-b3b3f6e5-5b87-4e79-b167-d4509e38838a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.219564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 871/1734] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:24.222803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9246005ab3b64f3db9fdf644a0b8be02', 'request_id': 'req-b3b3f6e5-5b87-4e79-b167-d4509e38838a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.255054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9246005ab3b64f3db9fdf644a0b8be02', 'request_id': 'req-b3b3f6e5-5b87-4e79-b167-d4509e38838a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.693475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] The token cache is being invalidate because project 9246005ab3b64f3db9fdf644a0b8be02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:24.693475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.693475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:24.693475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 864/1735] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:23 2022] DELETE /identity/v3/projects/9246005ab3b64f3db9fdf644a0b8be02 => generated 0 bytes in 1171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:24.696828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c545bfe-af65-4631-a180-960d193116e2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.697419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c545bfe-af65-4631-a180-960d193116e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.697419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c545bfe-af65-4631-a180-960d193116e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:24.720584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:24.731633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:24.799054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6c545bfe-af65-4631-a180-960d193116e2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:24.805855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c545bfe-af65-4631-a180-960d193116e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:24.937519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b3f6e5-5b87-4e79-b167-d4509e38838a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:24.945326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:24.947240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.948050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.948513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] PATH_INFO: `/v3/projects/16c56a99c1164f5c853884e74eb011a7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:24.950661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=16c56a99c1164f5c853884e74eb011a7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:24.953802 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:24.977825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 872/1736] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:24.986258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:24.986825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:24.987367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:25.023765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:25.025770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:25.080517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 873/1737] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:25.086753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:25.136233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a3fdd4-cb64-4958-bafa-22bdae4c89d2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:25.160325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:25.160325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:25.160325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:25.160325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:25.160833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:25.160833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:25.160833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:25.200781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 874/1738] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:25.205554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:25.242083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3e1a23-4e69-43ad-8314-ed6ab01c3400 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:25.263284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:25.263284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:25.263284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:25.263284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:25.265699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:25.265699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:25.413708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16c56a99c1164f5c853884e74eb011a7', 'request_id': 'req-7747901e-55e4-44d6-9a9c-11e199da01d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:25.447338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16c56a99c1164f5c853884e74eb011a7', 'request_id': 'req-7747901e-55e4-44d6-9a9c-11e199da01d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:25.459267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16c56a99c1164f5c853884e74eb011a7', 'request_id': 'req-7747901e-55e4-44d6-9a9c-11e199da01d4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:25.474243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 875/1739] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:25 2022] POST /identity/v3/projects => generated 403 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:25.482501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:25.516181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:25.654762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a0cdae-eba0-4f8d-822b-4a847c461e1f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:25.662739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:25.663491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:25.663929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:25.664231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:25.665631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:25.668904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:25.900750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] The token cache is being invalidate because project 16c56a99c1164f5c853884e74eb011a7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:25.901704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:25.903044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:25.933461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 876/1740] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:25 2022] POST /identity/v3/users => generated 302 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:25.935177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:25.951768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 865/1741] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:24 2022] DELETE /identity/v3/projects/16c56a99c1164f5c853884e74eb011a7 => generated 0 bytes in 1232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:25.957777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:25.958806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:25.962783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:26.005061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148749c6-91c4-4a16-95a6-5f4b4b702e57 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:26.020888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:26.021920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.022504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.022970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.023745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:26.025540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:26.071248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 877/1742] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:26.078509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:26.124321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85491d29-6d29-40f5-9985-93f188716b3d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:26.145192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:26.145437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.145833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.146302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.149241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:26.158290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:26.184012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 878/1743] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:26 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:26.186353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:26.197299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7747901e-55e4-44d6-9a9c-11e199da01d4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:26.216344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:26.219319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.220110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0613ed9c-f0ac-4a91-a905-ddf51dd690a5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:26.220402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.221081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] PATH_INFO: `/v3/users/8cb8aa5ce99d4ab98fed3e5b70fff4b7` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.228764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cb8aa5ce99d4ab98fed3e5b70fff4b7)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:26.234049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:26.240476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b28a6-f44e-47f6-82aa-ec8c78696980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:26.241824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b28a6-f44e-47f6-82aa-ec8c78696980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.243295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b28a6-f44e-47f6-82aa-ec8c78696980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.245263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b28a6-f44e-47f6-82aa-ec8c78696980 admin admin] PATH_INFO: `/v3/projects/cee33719e2f146a19247ee490f87a4e4/users/e2c2836e07ea4ea5ac675abf1ad37447/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.286926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b28a6-f44e-47f6-82aa-ec8c78696980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee33719e2f146a19247ee490f87a4e4, user_id=e2c2836e07ea4ea5ac675abf1ad37447, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:26.292673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b28a6-f44e-47f6-82aa-ec8c78696980 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:26.555927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:26.573744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 879/1744] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:26 2022] PUT /identity/v3/projects/cee33719e2f146a19247ee490f87a4e4/users/e2c2836e07ea4ea5ac675abf1ad37447/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:26.590558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:26.590851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:26.591126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:26.636158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] MFA Rules not processed for user `e2c2836e07ea4ea5ac675abf1ad37447`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:26.637606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:26.817780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 880/1745] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:26 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:26.820901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:26.924004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cb8aa5ce99d4ab98fed3e5b70fff4b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.927380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cb8aa5ce99d4ab98fed3e5b70fff4b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.995443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cb8aa5ce99d4ab98fed3e5b70fff4b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.995443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cb8aa5ce99d4ab98fed3e5b70fff4b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:26.995443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c6a4c284-5302-4bb0-bf94-e286f93bdbb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cb8aa5ce99d4ab98fed3e5b70fff4b7'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:27.035278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d6fba2-d9b2-4da7-83cc-d7fb33388b1b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.035278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.035278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.035278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.036085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.036085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:27.036085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:27.189087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 866/1746] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:25 2022] DELETE /identity/v3/users/8cb8aa5ce99d4ab98fed3e5b70fff4b7 => generated 0 bytes in 1233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:27.191694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c593ce-47c8-4e79-889e-d854addf4538 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.192143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c593ce-47c8-4e79-889e-d854addf4538 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.192561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c593ce-47c8-4e79-889e-d854addf4538 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.194194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 867/1747] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:31:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:27.238975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.239517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.239964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.267573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 881/1748] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:26 2022] POST /identity/v3/projects => generated 405 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:27.272941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:27.281993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:27.305108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:27.306330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:27.430665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efe7e6c-45b6-418b-bfae-9a650b028260 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.431784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 868/1749] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:31:27 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:27.437464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.459153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.459581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.462303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.463115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:27.465201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:27.726080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:27.727902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:27.766889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 882/1750] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:27 2022] POST /identity/v3/users => generated 309 bytes in 495 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:27.771631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:27.821572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b406e0d4-21c5-42c4-b858-ab86f5dd5638 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.823859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08fa290-68cb-4c2c-97a4-c6bcaef07719 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.890143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.897919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.900081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.900577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.901518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:27.903756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.904419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.904704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.904976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.906809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:27.938293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 883/1751] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:27 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:27.942229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:27.967782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf7ae20-88b7-49dc-98a3-bce1930a88d3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:27.975140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:27.977527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:27.978094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:27.978820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] PATH_INFO: `/v3/projects/0213dfca64f44161bdd353b5a76052cd/users/8fef70ff6a754bbda03b19639cde47e1/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:27.983821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:27.995567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.005096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0213dfca64f44161bdd353b5a76052cd, user_id=8fef70ff6a754bbda03b19639cde47e1, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.009429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.069547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 869/1752] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:27 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:28.150685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 884/1753] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:27 2022] PUT /identity/v3/projects/0213dfca64f44161bdd353b5a76052cd/users/8fef70ff6a754bbda03b19639cde47e1/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:28.151492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.193476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd2c2be-5579-46c3-a304-b8c22dac6eaa service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.194999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.196307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:28.417963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.420543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.421665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.422169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.422978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b440c889-b449-4565-b04b-f6da9bdd5e96 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.423311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.438738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.439733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.440808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.441551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] PATH_INFO: `/v3/projects/3a12407f664448bc864f6e8638f1a107` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.443060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.443526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a12407f664448bc864f6e8638f1a107)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.446896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.479129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 870/1754] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:28 2022] GET /identity/v3/roles => generated 1420 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:28.485405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.497825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8145b5-d0e8-4d36-9fd1-bfbb4296c79d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:28.509056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0944610a-a29c-44d2-9b6b-14a15014f238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:28.510848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944610a-a29c-44d2-9b6b-14a15014f238 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.512394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944610a-a29c-44d2-9b6b-14a15014f238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.513611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0944610a-a29c-44d2-9b6b-14a15014f238 admin admin] PATH_INFO: `/v3/projects/0213dfca64f44161bdd353b5a76052cd/users/8fef70ff6a754bbda03b19639cde47e1/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.517555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944610a-a29c-44d2-9b6b-14a15014f238 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0213dfca64f44161bdd353b5a76052cd, user_id=8fef70ff6a754bbda03b19639cde47e1, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:28.522384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0944610a-a29c-44d2-9b6b-14a15014f238 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:28.527799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:28.560558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 871/1755] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:28 2022] PUT /identity/v3/projects/0213dfca64f44161bdd353b5a76052cd/users/8fef70ff6a754bbda03b19639cde47e1/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:28.563719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:28.565804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:28.571837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:28.638557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] MFA Rules not processed for user `8fef70ff6a754bbda03b19639cde47e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:28.640142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:28.688538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a12407f664448bc864f6e8638f1a107', 'request_id': 'req-5ccd85c6-3737-4091-b00c-cf5321104eb9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:28.727199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a12407f664448bc864f6e8638f1a107', 'request_id': 'req-5ccd85c6-3737-4091-b00c-cf5321104eb9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:28.810575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 872/1756] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:28 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:28.812100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.814227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:28.906767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a12407f664448bc864f6e8638f1a107', 'request_id': 'req-5ccd85c6-3737-4091-b00c-cf5321104eb9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:28.944541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] The token cache is being invalidate because project 3a12407f664448bc864f6e8638f1a107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:28.944541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:28.944541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:28.952637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 885/1757] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:28 2022] DELETE /identity/v3/projects/3a12407f664448bc864f6e8638f1a107 => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:28.954157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:28.959290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:29.000650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b521e3-71f6-4aa3-a0db-adf636f634d5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.136107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccd85c6-3737-4091-b00c-cf5321104eb9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.153310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.153310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.153310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.153310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] PATH_INFO: `/v3/users/83356e7f648442b590d70dd8e736e3f1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.153785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.157268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=83356e7f648442b590d70dd8e736e3f1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.158161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.160159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.161985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.183046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:29.197931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.307669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.329521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69fb221c-6857-45d5-afc6-405614201f70 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.353823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 886/1758] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:28 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:29.361635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950e485-68b4-4bb9-b756-57348b4416d4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.362560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950e485-68b4-4bb9-b756-57348b4416d4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.363295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8950e485-68b4-4bb9-b756-57348b4416d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.435766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8950e485-68b4-4bb9-b756-57348b4416d4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:29.435766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8950e485-68b4-4bb9-b756-57348b4416d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:29.463347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 887/1759] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:29.473306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace181c-4466-4752-a04c-713600f1721a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.473543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace181c-4466-4752-a04c-713600f1721a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.473729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace181c-4466-4752-a04c-713600f1721a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.539314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8ace181c-4466-4752-a04c-713600f1721a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:29.540705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8ace181c-4466-4752-a04c-713600f1721a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:29.574932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 888/1760] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:29.576269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace181c-4466-4752-a04c-713600f1721a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.588373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace181c-4466-4752-a04c-713600f1721a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.594190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.595181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.595578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.595942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.597658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:29.597658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.599925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.617491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 889/1761] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:29.621834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.635076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:29.648071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c3ccdb-b650-4831-9533-091bd101dba6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:29.652931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:29.653643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:29.653985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:29.654340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:29.654970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:29.657481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:29.784492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 890/1762] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:29 2022] POST /identity/v3/projects => generated 401 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:29.787876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:29.806041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83356e7f648442b590d70dd8e736e3f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:29.806840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83356e7f648442b590d70dd8e736e3f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:29.858077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83356e7f648442b590d70dd8e736e3f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:29.858906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83356e7f648442b590d70dd8e736e3f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.126689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83356e7f648442b590d70dd8e736e3f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:30.132316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3758785d-dc27-400e-bc99-59ba365aec51 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:30.132316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd7e7ec-69ae-4ff5-9ca9-8c41238dab8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:30.132316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7e7ec-69ae-4ff5-9ca9-8c41238dab8b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.132316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7e7ec-69ae-4ff5-9ca9-8c41238dab8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.133401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd7e7ec-69ae-4ff5-9ca9-8c41238dab8b admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.133401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7e7ec-69ae-4ff5-9ca9-8c41238dab8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:30.133401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd7e7ec-69ae-4ff5-9ca9-8c41238dab8b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:30.456534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 873/1763] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:28 2022] DELETE /identity/v3/users/83356e7f648442b590d70dd8e736e3f1 => generated 0 bytes in 1632 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:30.456534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:30.456534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:30.461058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 891/1764] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:29 2022] POST /identity/v3/users => generated 301 bytes in 674 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:30.463746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.465873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.466059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.559849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:30.561356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:30.697064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29cf7e70-e168-4dca-b64c-8e48aee60e6e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:30.697981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:30.699779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.701961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.702626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] PATH_INFO: `/v3/users/e2c2836e07ea4ea5ac675abf1ad37447` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.706738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2c2836e07ea4ea5ac675abf1ad37447)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:30.709903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:30.726984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:30.751591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 892/1765] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:30.751591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:30.751591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:30.773921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b64277e-e499-472f-b210-a8250b2f7d36 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:30.787861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbe8c30-f85a-4ad7-a86f-adc739607133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:30.788664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8c30-f85a-4ad7-a86f-adc739607133 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.789073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8c30-f85a-4ad7-a86f-adc739607133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.789443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbe8c30-f85a-4ad7-a86f-adc739607133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.790271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe8c30-f85a-4ad7-a86f-adc739607133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:30.792184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbe8c30-f85a-4ad7-a86f-adc739607133 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:30.817141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 893/1766] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:30.819438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6e94-432c-414a-8159-4e48317fa2e4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:30.819981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6e94-432c-414a-8159-4e48317fa2e4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:30.820685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93d6e94-432c-414a-8159-4e48317fa2e4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:30.823295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 894/1767] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:31:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:30.824744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6e94-432c-414a-8159-4e48317fa2e4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:30.828000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d93d6e94-432c-414a-8159-4e48317fa2e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:31.006787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93d6e94-432c-414a-8159-4e48317fa2e4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:31.028501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:31.029511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.030005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.030436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.044287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:31.045993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:31.059549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 895/1768] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:31.060524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:31.072802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc651ca1-385b-490b-8e8d-3a62a238ca43 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:31.088592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba5293b-9086-4447-99f8-773a2380788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:31.089367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5293b-9086-4447-99f8-773a2380788e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.089684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5293b-9086-4447-99f8-773a2380788e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.090013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba5293b-9086-4447-99f8-773a2380788e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.090753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5293b-9086-4447-99f8-773a2380788e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:31.092888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba5293b-9086-4447-99f8-773a2380788e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:31.111367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 896/1769] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:31.113946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.114413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.114846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.144648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:31.146287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:31.259074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 897/1770] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:31:31 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:31.259074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:31.284574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd866f-ddbb-4a33-b1f9-06437eb29fe4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:31.298339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:31.299152 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.299972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.300246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] PATH_INFO: `/v3/projects/0014fc7a4cde4dbbbf33a234378c79dd/users/ef6fa31a87c84df8bb125426ca479572/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.338741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0014fc7a4cde4dbbbf33a234378c79dd, user_id=ef6fa31a87c84df8bb125426ca479572, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:31.341497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:31.388072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2c2836e07ea4ea5ac675abf1ad37447'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.390027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2c2836e07ea4ea5ac675abf1ad37447'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.433766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2c2836e07ea4ea5ac675abf1ad37447'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.436048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2c2836e07ea4ea5ac675abf1ad37447'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.458984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d7d81c98-e14b-44e0-b96f-496173d6a9d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2c2836e07ea4ea5ac675abf1ad37447'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:31.667473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 898/1771] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:31 2022] PUT /identity/v3/projects/0014fc7a4cde4dbbbf33a234378c79dd/users/ef6fa31a87c84df8bb125426ca479572/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:31.670594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:31.684327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:31.695514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 874/1772] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:30 2022] DELETE /identity/v3/users/e2c2836e07ea4ea5ac675abf1ad37447 => generated 0 bytes in 1248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:31.697538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75501538-728d-428f-99fb-4ef8c9ba5c9b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.698063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75501538-728d-428f-99fb-4ef8c9ba5c9b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.698541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75501538-728d-428f-99fb-4ef8c9ba5c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.767028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-75501538-728d-428f-99fb-4ef8c9ba5c9b None None] MFA Rules not processed for user `ef6fa31a87c84df8bb125426ca479572`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:31.769732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-75501538-728d-428f-99fb-4ef8c9ba5c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:31.962604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 875/1773] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:31 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:31.962604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:31.962604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:31.962604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:31.991475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65567257-7ba6-486c-bc3e-949f6d8efa24 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:32.011568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:32.014036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:32.185990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 876/1774] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:32.188674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:32.189388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:32.189563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:32.192221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:32.192567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:32.192976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:32.197689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:32.227516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d593ceb-702c-4577-8875-ab57019030e5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:32.243384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:32.257382 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:32.263091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:32.264459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:32.267372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:32.269384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:32.278750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:32.292477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:32.366270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 899/1775] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 683 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:32.366270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:32.380682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:32.599329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 877/1776] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:32 2022] POST /identity/v3/projects => generated 401 bytes in 412 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:32.605839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:32.613088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:32.706766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46a7fe1-83ed-4f81-9fad-d5eae6811601 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:32.869270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ff697b-924c-4d07-92c5-547c63e68f11 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:32.879668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:32.881772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:32.885151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:32.885151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:32.891910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:32.894182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:32.894732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:32.895258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] PATH_INFO: `/v3/users/5e17763e0a85482aaf468a7c2c3e9776` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:32.898833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e17763e0a85482aaf468a7c2c3e9776)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:32.904509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:32.906872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:32.922584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:32.988516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 900/1777] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 625 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:32.989830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.007589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f82599-c07c-4f4c-9468-f8cfca37790e service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.015874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.021547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.021547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.021547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.021547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.021547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.527754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 901/1778] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:32 2022] POST /identity/v3/users => generated 307 bytes in 533 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:33.527754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.544295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d637dff-4e6e-49c8-92c6-6f9000e5b9af admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.554091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.559766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.560840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.561238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.562044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.564194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.587761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 902/1779] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:33.589109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:33.607207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d0840a-ab6e-4e78-8433-9908de916c01 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:33.623881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e438f-270d-4505-a588-05bf852da968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:33.625488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e438f-270d-4505-a588-05bf852da968 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:33.625488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e438f-270d-4505-a588-05bf852da968 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:33.626051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e438f-270d-4505-a588-05bf852da968 admin admin] PATH_INFO: `/v3/projects/ab02b558fdbe428a92d7fd426d20f8fc/users/03fc47eacc3d4d509d0876512532fdfd/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:33.668023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e438f-270d-4505-a588-05bf852da968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab02b558fdbe428a92d7fd426d20f8fc, user_id=03fc47eacc3d4d509d0876512532fdfd, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:33.685619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e438f-270d-4505-a588-05bf852da968 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:33.728558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e17763e0a85482aaf468a7c2c3e9776'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.730126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e17763e0a85482aaf468a7c2c3e9776'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.788137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e17763e0a85482aaf468a7c2c3e9776'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.789334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e17763e0a85482aaf468a7c2c3e9776'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:33.804470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e17763e0a85482aaf468a7c2c3e9776'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:34.137677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 878/1780] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:32 2022] DELETE /identity/v3/users/5e17763e0a85482aaf468a7c2c3e9776 => generated 0 bytes in 1534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:34.138569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.144380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:34.148426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 903/1781] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:33 2022] PUT /identity/v3/projects/ab02b558fdbe428a92d7fd426d20f8fc/users/03fc47eacc3d4d509d0876512532fdfd/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:34.150754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.151254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.152090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.182029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:34.183742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:34.275311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550eb98f-4114-4016-a9fe-0de60dbd06b4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.282627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.286005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 904/1782] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:34.286157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.286254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.286333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.286333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] PATH_INFO: `/v3/projects/250c10ecd98848ebbb2561dfe9a5dafa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.286880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:34.298745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] RBAC: Authorizing `identity:delete_project(project_id=250c10ecd98848ebbb2561dfe9a5dafa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.306558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.311861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ac6458-05b0-4995-a45c-5bcf91a723df None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.321327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.322431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.322937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.323508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.335736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.338390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.426970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 905/1783] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:34.426970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.426970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:34.447166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9955e9bd-14ca-4b4f-b0ad-1e33e248b505 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.469447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.470293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.470479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.470774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.515607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.517312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.555217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 906/1784] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:34.556131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.579238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279d0745-23d3-4e5c-b31c-e581cab1fc0f service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:34.590197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1af91b-8d24-4390-bdca-280b96c71b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:34.590892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1af91b-8d24-4390-bdca-280b96c71b51 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.591176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1af91b-8d24-4390-bdca-280b96c71b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.591505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1af91b-8d24-4390-bdca-280b96c71b51 admin admin] PATH_INFO: `/v3/projects/ab02b558fdbe428a92d7fd426d20f8fc/users/03fc47eacc3d4d509d0876512532fdfd/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.597131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1af91b-8d24-4390-bdca-280b96c71b51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab02b558fdbe428a92d7fd426d20f8fc, user_id=03fc47eacc3d4d509d0876512532fdfd, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:34.600415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1af91b-8d24-4390-bdca-280b96c71b51 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:34.631803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 907/1785] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:34 2022] PUT /identity/v3/projects/ab02b558fdbe428a92d7fd426d20f8fc/users/03fc47eacc3d4d509d0876512532fdfd/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:34.642588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:34.643641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:34.644150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:34.697278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] MFA Rules not processed for user `03fc47eacc3d4d509d0876512532fdfd`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:34.703934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:34.761553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:34.832552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '250c10ecd98848ebbb2561dfe9a5dafa', 'request_id': 'req-6856a467-2d5f-44e2-b544-55b96e6f998f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:34.871769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '250c10ecd98848ebbb2561dfe9a5dafa', 'request_id': 'req-6856a467-2d5f-44e2-b544-55b96e6f998f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:34.922817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '250c10ecd98848ebbb2561dfe9a5dafa', 'request_id': 'req-6856a467-2d5f-44e2-b544-55b96e6f998f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:34.928423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 908/1786] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:34 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:34.928423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:34.928423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:35.047569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] The token cache is being invalidate because project 250c10ecd98848ebbb2561dfe9a5dafa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:35.048214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.049426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.055889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 879/1787] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:34 2022] DELETE /identity/v3/projects/250c10ecd98848ebbb2561dfe9a5dafa => generated 0 bytes in 914 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:35.057403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:35.059954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:35.086192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d084574e-ee79-4b64-bb40-908e4396d363 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:35.238323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6856a467-2d5f-44e2-b544-55b96e6f998f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:35.239007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:35.239007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.239007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.239007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] PATH_INFO: `/v3/projects/cee33719e2f146a19247ee490f87a4e4` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.244892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee33719e2f146a19247ee490f87a4e4)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:35.248528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.257197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:35.261643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.262950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.263524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.269128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:35.427546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:35.430491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.445799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 880/1788] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:35 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:35.446976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:35.448512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:35.467312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d788580c-0956-4b56-9bce-8587f61d14fe service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:35.473342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:35.474214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:35.474664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:35.474985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] PATH_INFO: `/v3/users/a771f59cbd74436ea258781928f279d2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:35.476757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a771f59cbd74436ea258781928f279d2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:35.479837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:35.793070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee33719e2f146a19247ee490f87a4e4', 'request_id': 'req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.814532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee33719e2f146a19247ee490f87a4e4', 'request_id': 'req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.864152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a771f59cbd74436ea258781928f279d2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.865237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a771f59cbd74436ea258781928f279d2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.878840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a771f59cbd74436ea258781928f279d2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.879867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a771f59cbd74436ea258781928f279d2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.887943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a771f59cbd74436ea258781928f279d2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.938114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee33719e2f146a19247ee490f87a4e4', 'request_id': 'req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.951011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] The token cache is being invalidate because project cee33719e2f146a19247ee490f87a4e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:35.951689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.952047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:35.953771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 909/1789] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:34 2022] DELETE /identity/v3/projects/cee33719e2f146a19247ee490f87a4e4 => generated 0 bytes in 1028 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:35.958042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:35.959510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:36.038482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9e39d-1b23-487f-9568-fd9d6174ae3d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:36.042655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:36.043288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:36.043611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:36.043891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:36.059099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:36.060637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 881/1790] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:35 2022] DELETE /identity/v3/users/a771f59cbd74436ea258781928f279d2 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:36.061545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:36.063187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:36.186123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:36.189339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:36.192775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9b2241-51ca-49fb-9154-5ffc8fede50e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:36.199103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:36.199890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:36.200352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:36.201261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] PATH_INFO: `/v3/users/8fef70ff6a754bbda03b19639cde47e1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:36.202586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 910/1791] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:35 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:36.203601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:36.204855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fef70ff6a754bbda03b19639cde47e1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:36.204962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:36.207256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:36.289564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d015cf-a31a-410d-882a-46b41a604508 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:36.297087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:36.298054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:36.299552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:36.303361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] PATH_INFO: `/v3/projects/2d975a3d83944540a82e8114322c3d27` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:36.303361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d975a3d83944540a82e8114322c3d27)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:36.304740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:36.578240 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d975a3d83944540a82e8114322c3d27', 'request_id': 'req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.580683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fef70ff6a754bbda03b19639cde47e1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.581514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fef70ff6a754bbda03b19639cde47e1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.585274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d975a3d83944540a82e8114322c3d27', 'request_id': 'req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.593770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fef70ff6a754bbda03b19639cde47e1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.594472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fef70ff6a754bbda03b19639cde47e1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.602425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-7e706a48-a1f9-42a3-9e38-43267182bd5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fef70ff6a754bbda03b19639cde47e1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.743080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d975a3d83944540a82e8114322c3d27', 'request_id': 'req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.771181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] The token cache is being invalidate because project 2d975a3d83944540a82e8114322c3d27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:36.771763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.772915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:36.776270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 911/1792] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:36 2022] DELETE /identity/v3/projects/2d975a3d83944540a82e8114322c3d27 => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:36.778094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:36.780362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:36.853354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f8e0ca-ae71-49fb-b0c9-4346c1fbd31c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:36.858166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:36.858815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:36.859089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:36.859387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] PATH_INFO: `/v3/projects/c111e05d7a06499682d04040c7367d8b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:36.860513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c111e05d7a06499682d04040c7367d8b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:36.862763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:36.945031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 882/1793] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:36 2022] DELETE /identity/v3/users/8fef70ff6a754bbda03b19639cde47e1 => generated 0 bytes in 884 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:36.946572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f974-213a-4d67-a678-9d9d5182f02f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:36.946895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f974-213a-4d67-a678-9d9d5182f02f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:36.947361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6f974-213a-4d67-a678-9d9d5182f02f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:36.972055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a5e6f974-213a-4d67-a678-9d9d5182f02f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:36.973950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a5e6f974-213a-4d67-a678-9d9d5182f02f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:37.053923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 883/1794] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:37.055831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.056246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.056613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.078551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:37.079730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:37.088693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 884/1795] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:37.093196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.104432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ba3747-55f7-4e6b-b79c-19a1739ccbca None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.109648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.110356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.110665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.110945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.112226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:37.112531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.114467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.126777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 885/1796] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:31:37.137227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.143717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf55b46-86a9-4d3c-9520-b54dbb148e2f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.149391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.150095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.150383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.150656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.151464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.153425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.193915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c111e05d7a06499682d04040c7367d8b', 'request_id': 'req-e94dbf6e-7615-4e38-9824-58428cff0d6e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:37.199357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c111e05d7a06499682d04040c7367d8b', 'request_id': 'req-e94dbf6e-7615-4e38-9824-58428cff0d6e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:37.277471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 886/1797] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/projects => generated 373 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:37.290264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.317664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c111e05d7a06499682d04040c7367d8b', 'request_id': 'req-e94dbf6e-7615-4e38-9824-58428cff0d6e'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:37.337335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] The token cache is being invalidate because project c111e05d7a06499682d04040c7367d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:37.341164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:37.342401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e94dbf6e-7615-4e38-9824-58428cff0d6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:37.345146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 912/1798] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:36 2022] DELETE /identity/v3/projects/c111e05d7a06499682d04040c7367d8b => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:37.353712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5819f20-dad6-4596-bea3-7172906e6989 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.353852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5819f20-dad6-4596-bea3-7172906e6989 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.354336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5819f20-dad6-4596-bea3-7172906e6989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.396521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d5819f20-dad6-4596-bea3-7172906e6989 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:37.397925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d5819f20-dad6-4596-bea3-7172906e6989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:37.404770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc52976-460f-43d8-85ae-a7e241dc4ebf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.414069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 913/1799] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:37.414270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.415032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.415317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.415591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.429714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.429714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.430003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.430003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.430003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.451173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:37.452726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:37.470114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 914/1800] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:37.474632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.489687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce57bf61-0b64-4b27-8390-e94e0dbb832e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.496516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.499286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.499286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.499286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.499286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:37.499595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.501579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.518379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 915/1801] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:37.522770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.537562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f776e5d2-081c-47e6-88e3-4b6b8859296e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.544136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.545221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.545603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.545952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.546656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.553149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.597383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 887/1802] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/users => generated 287 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:37.605189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.606953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:37.620547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f51559-8ddf-40da-bcf5-d58a96da3d0b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.626026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.626660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.627595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.627854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.633113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.635159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.648450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 888/1803] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:37.651403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.662054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92893f0b-939f-4088-86ad-d874ed65c9a6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.666999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.667625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.668192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.668901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.670546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.686853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 889/1804] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:37 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:37.689278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.701483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c0bdc9-9dac-49a2-a20d-3f92118fd294 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.706827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.707589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.707924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.708388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] PATH_INFO: `/v3/projects/3740344ed3d44994a8b3ef9cdbb69bf5/users/bf7c7782806442f2971a6b583a66b1c2/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.727621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 916/1805] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/projects => generated 375 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:37.729451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.732320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:37.738891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3740344ed3d44994a8b3ef9cdbb69bf5, user_id=bf7c7782806442f2971a6b583a66b1c2, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.741492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.811516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a919b3-be9e-4cb5-ae48-3f56b86c12e9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.817325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.817967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.818269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.818546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] PATH_INFO: `/v3/projects/0213dfca64f44161bdd353b5a76052cd` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.819670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] RBAC: Authorizing `identity:delete_project(project_id=0213dfca64f44161bdd353b5a76052cd)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.821630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.860522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 890/1806] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:37 2022] PUT /identity/v3/projects/3740344ed3d44994a8b3ef9cdbb69bf5/users/bf7c7782806442f2971a6b583a66b1c2/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:37.861823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:37.931235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8085448-2c19-4b3f-95db-228c69d293f6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:37.935911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c2051-4b3d-4261-804b-a2e3d38f39e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:37.936527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c2051-4b3d-4261-804b-a2e3d38f39e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:37.936900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c2051-4b3d-4261-804b-a2e3d38f39e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:37.937136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c2051-4b3d-4261-804b-a2e3d38f39e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:37.937768 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c2051-4b3d-4261-804b-a2e3d38f39e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:37.939853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c2051-4b3d-4261-804b-a2e3d38f39e2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:37.992863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0213dfca64f44161bdd353b5a76052cd', 'request_id': 'req-48aaf97f-481f-4342-a7f2-a082a360906c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:37.998757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0213dfca64f44161bdd353b5a76052cd', 'request_id': 'req-48aaf97f-481f-4342-a7f2-a082a360906c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:38.056633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 891/1807] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:37 2022] POST /identity/v3/users => generated 288 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:38.056633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f5c0-90ab-4d93-a256-94d8aef22510 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.056633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f5c0-90ab-4d93-a256-94d8aef22510 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.056633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2174f5c0-90ab-4d93-a256-94d8aef22510 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.086371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2174f5c0-90ab-4d93-a256-94d8aef22510 None None] MFA Rules not processed for user `bf7c7782806442f2971a6b583a66b1c2`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:38.087673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2174f5c0-90ab-4d93-a256-94d8aef22510 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:38.133200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0213dfca64f44161bdd353b5a76052cd', 'request_id': 'req-48aaf97f-481f-4342-a7f2-a082a360906c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:38.150213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] The token cache is being invalidate because project 0213dfca64f44161bdd353b5a76052cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:38.150914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:38.151792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:38.154097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 917/1808] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:37 2022] DELETE /identity/v3/projects/0213dfca64f44161bdd353b5a76052cd => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:38.154983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.156581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:38.168427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 892/1809] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:38.173053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.173350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.173627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.194601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:38.200580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:38.239601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48aaf97f-481f-4342-a7f2-a082a360906c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.244942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.245730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.246020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.246417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.247104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.248818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.259572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 918/1810] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:38.260479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.262189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:38.264450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 893/1811] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:38.265338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.276088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52f57ea-d83d-4216-934f-9303ad41cb9e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.276413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16972514-9cf9-4d4d-b158-813260d57b01 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.284125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9f9469-2753-4df7-974f-b275450e95ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.284743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f9469-2753-4df7-974f-b275450e95ea admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.285040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f9469-2753-4df7-974f-b275450e95ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.285324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9f9469-2753-4df7-974f-b275450e95ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.285496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.285975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f9469-2753-4df7-974f-b275450e95ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.286143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.286398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.286667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.287403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.287565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9f9469-2753-4df7-974f-b275450e95ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.289491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.298108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 894/1812] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:38.299372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.299656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.299931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.320872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:38.322124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:38.331176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 895/1813] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:38.332062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.341799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fdc0d6-9f18-4eb6-9317-5649655c922c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.347299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.347977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.348258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.348532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] PATH_INFO: `/v3/projects/7020dacd068a42158f3dbb5efc28b007/users/7fb958fb394443288f24b5dcda37ee7b/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.363756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7020dacd068a42158f3dbb5efc28b007, user_id=7fb958fb394443288f24b5dcda37ee7b, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.366469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.465385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 919/1814] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/projects => generated 373 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:38.466328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.510381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 896/1815] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:38 2022] PUT /identity/v3/projects/7020dacd068a42158f3dbb5efc28b007/users/7fb958fb394443288f24b5dcda37ee7b/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:38.511327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.535327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c02243a-e8bc-473a-b59e-7c00456230de admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.579642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3601db14-2ab8-4dcc-874d-fa75a3687d22 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.585572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.586331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.586525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.586806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.587568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.590038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.610470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.611442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.611855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.612141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.613049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:38.613371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.614995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7749851-abbd-4d8a-9373-4b84f5500a74 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.627338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 920/1816] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:38.628663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d480-6108-4017-bf09-37f085e65005 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.628948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d480-6108-4017-bf09-37f085e65005 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.629249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d456d480-6108-4017-bf09-37f085e65005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.664574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d456d480-6108-4017-bf09-37f085e65005 None None] MFA Rules not processed for user `7fb958fb394443288f24b5dcda37ee7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:38.665889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d456d480-6108-4017-bf09-37f085e65005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:38.741560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 921/1817] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:38.742473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d480-6108-4017-bf09-37f085e65005 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.753701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d456d480-6108-4017-bf09-37f085e65005 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.761530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.762173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.762454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.762736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.763413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.765529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.769160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:38.778217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 897/1818] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/users => generated 293 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:38.778217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.791665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3543b558-6ef5-40c9-a090-8f4d75f858d9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.794659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:38.795278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:38.795565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:38.795844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:38.796486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:38.798476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:38.884129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 922/1819] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/projects => generated 363 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:38.885284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:38.950478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a072be8-89bf-4ead-a4b3-2a8d8a509e6f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:38.960355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 898/1820] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/projects => generated 377 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:38.960355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.018460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.019102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.020207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.020207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.020402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.021569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335dc8aa-b0fb-4c36-aaf1-c4371e6d0408 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.022075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.026388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.026994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.027276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.027551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.028221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.030289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.032891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 923/1821] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:38 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:39.033668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.043837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78cca84-1e80-4fb7-81c4-2f2be05bacec admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.048552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.049686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.049965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.050237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.050889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.052788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.161989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 899/1822] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:38 2022] POST /identity/v3/users => generated 282 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:39.162813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.172889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5fec06-cd82-4098-a7e1-6b8411ba7979 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.179056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.179725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.180003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.180272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] PATH_INFO: `/v3/projects/5d030540d70846f59b10ce57f31ad0dc/users/c906e0990ba14e688ab3c0b39449b42f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.196967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 924/1823] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:39 2022] POST /identity/v3/users => generated 295 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:39.197941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.199899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d030540d70846f59b10ce57f31ad0dc, user_id=c906e0990ba14e688ab3c0b39449b42f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.208887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.209329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fb176b-92c5-491e-9a31-7be75ebbb2f7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.214224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.214836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.215118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.215393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.216047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.217768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.230309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 925/1824] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:39.231299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.242507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1216fafe-7654-46b2-b053-b33e52f3d3ba admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.247471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.248004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.248285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.248561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.249230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.250936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.261801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 926/1825] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:39.262797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.274743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b864c2c2-a5f5-45c7-97d3-a718b8bbb4e3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.279811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.280038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.280341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.280613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.281272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.282868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.293204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 927/1826] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:39.294067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.304806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9abcc7f-0483-42dd-a7ea-bdf40d7d66a5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.307914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 900/1827] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:39 2022] PUT /identity/v3/projects/5d030540d70846f59b10ce57f31ad0dc/users/c906e0990ba14e688ab3c0b39449b42f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:39.312324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.367718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.368399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.368681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.368959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] PATH_INFO: `/v3/projects/bfaae4ceaee54d1ba21f172b6a7fc9ac/users/f2e941cc771d4f83b94faaf0c78dbbb8/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.384232 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfaae4ceaee54d1ba21f172b6a7fc9ac, user_id=f2e941cc771d4f83b94faaf0c78dbbb8, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.386939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.390972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d12c6b-38c1-4a73-abd3-d5265f7c6121 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.396279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.396981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.397283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.397557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] PATH_INFO: `/v3/projects/27f3464b928a4870984f70699ddf0292/users/49d8b06354dd459ebd7eb8f66b2abdb3/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.414456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f3464b928a4870984f70699ddf0292, user_id=49d8b06354dd459ebd7eb8f66b2abdb3, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.417123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.496352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 928/1828] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:39 2022] PUT /identity/v3/projects/bfaae4ceaee54d1ba21f172b6a7fc9ac/users/f2e941cc771d4f83b94faaf0c78dbbb8/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:39.497938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.569198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8e3768-4e5d-4fe9-981d-53612a3952fb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.573033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 901/1829] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:39 2022] PUT /identity/v3/projects/27f3464b928a4870984f70699ddf0292/users/49d8b06354dd459ebd7eb8f66b2abdb3/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:39.573732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.634857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e24cacf-7971-4be9-b294-7515de8447c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.634857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24cacf-7971-4be9-b294-7515de8447c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.634857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24cacf-7971-4be9-b294-7515de8447c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.634857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e24cacf-7971-4be9-b294-7515de8447c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.635445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24cacf-7971-4be9-b294-7515de8447c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.637328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e24cacf-7971-4be9-b294-7515de8447c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.659038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cf1ba1-5d68-4a79-8f72-71535d3be644 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.672628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 929/1830] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:39.673590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec32b-b723-44b0-a91b-455bdf1ad6c0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.674472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec32b-b723-44b0-a91b-455bdf1ad6c0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.676961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.677699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.686223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.686223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.686223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.686544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaec32b-b723-44b0-a91b-455bdf1ad6c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.687532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.714738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 902/1831] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:39 2022] GET /identity/v3/roles => generated 1420 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:39.715856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.739214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aecd7b-46d8-4508-ac8d-f25f4cf7485a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.743580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6eaec32b-b723-44b0-a91b-455bdf1ad6c0 None None] MFA Rules not processed for user `49d8b06354dd459ebd7eb8f66b2abdb3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:39.744829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6eaec32b-b723-44b0-a91b-455bdf1ad6c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:39.746656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.747665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.748098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.748542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] PATH_INFO: `/v3/projects/5d030540d70846f59b10ce57f31ad0dc/users/c906e0990ba14e688ab3c0b39449b42f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.751787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d030540d70846f59b10ce57f31ad0dc, user_id=c906e0990ba14e688ab3c0b39449b42f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.755832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.768875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 903/1832] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:39 2022] PUT /identity/v3/projects/5d030540d70846f59b10ce57f31ad0dc/users/c906e0990ba14e688ab3c0b39449b42f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:39.774003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:39.840180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 930/1833] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:39 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:39.841913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fa152-b0a7-4de1-a176-d75133d07bc1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.842300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fa152-b0a7-4de1-a176-d75133d07bc1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.842673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80fa152-b0a7-4de1-a176-d75133d07bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.899756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c80fa152-b0a7-4de1-a176-d75133d07bc1 None None] MFA Rules not processed for user `c906e0990ba14e688ab3c0b39449b42f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:39.901206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c80fa152-b0a7-4de1-a176-d75133d07bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:39.902307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8411166f-2e00-4b6e-9d0c-45a7c6eecf2a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:39.907306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:39.907989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:39.908560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:39.908839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] PATH_INFO: `/v3/projects/bfaae4ceaee54d1ba21f172b6a7fc9ac/users/f2e941cc771d4f83b94faaf0c78dbbb8/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:39.911021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfaae4ceaee54d1ba21f172b6a7fc9ac, user_id=f2e941cc771d4f83b94faaf0c78dbbb8, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:39.913907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:39.927973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 904/1834] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:39 2022] PUT /identity/v3/projects/bfaae4ceaee54d1ba21f172b6a7fc9ac/users/f2e941cc771d4f83b94faaf0c78dbbb8/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:39.928965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.009710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 931/1835] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:39 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:40.009710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.009710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.009710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.020479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86ae7d5-93c9-400b-921e-dbc752ca162d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.025936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.026554 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.026835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.027104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.027728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.029653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.051161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] MFA Rules not processed for user `f2e941cc771d4f83b94faaf0c78dbbb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:40.052385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:40.123425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 905/1836] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:39 2022] POST /identity/v3/projects => generated 363 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:40.129544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.131104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 932/1837] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:40 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:40.196099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a4b3a8-3757-4bdf-a8f0-0967e3f34fad admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.200861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.201566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.201846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.202118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.202757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.204695 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.379625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 906/1838] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:40 2022] POST /identity/v3/users => generated 288 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:40.384751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.397106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21ec83a-744c-48a2-b45d-49cdbe8b06d2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.402833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.403516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.403797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.404068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.404712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.406373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.418512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.418757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 933/1839] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:40.420360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:40.424368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.434957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f5fa7-bbd5-4834-a265-0f793035bd3c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.440836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.441147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.441416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.441871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] PATH_INFO: `/v3/projects/5ba9b2ff218145d38bf1485a55ef654e/users/b1f498cc67d34da5880626afc9eabc51/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.459227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9b2ff218145d38bf1485a55ef654e, user_id=b1f498cc67d34da5880626afc9eabc51, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.462061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.502226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cff1404-ca02-49c9-a9f0-0b2e8600d25c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.507205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.507826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.508103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.508386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.580946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 934/1840] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:40 2022] PUT /identity/v3/projects/5ba9b2ff218145d38bf1485a55ef654e/users/b1f498cc67d34da5880626afc9eabc51/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:40.589146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.591542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.594443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.615460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 907/1841] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:40.659145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968ddde9-ba13-431e-8dbe-9685c13fc1db admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.665576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:40.668538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.670397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:40.671175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.671897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.672243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.672604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.673393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.676046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.684233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea5f936-1dcb-4473-9b42-bb239e2264fe service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.690596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 935/1842] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:40.691909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:40.702591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8af791e-6d31-4694-86c5-4cab5ece7031 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:40.708268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b81069-c92f-4b5e-bbe1-a10bf7c7947d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.708968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b81069-c92f-4b5e-bbe1-a10bf7c7947d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.709285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b81069-c92f-4b5e-bbe1-a10bf7c7947d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.709581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b81069-c92f-4b5e-bbe1-a10bf7c7947d admin admin] PATH_INFO: `/v3/projects/5ba9b2ff218145d38bf1485a55ef654e/users/b1f498cc67d34da5880626afc9eabc51/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.711790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b81069-c92f-4b5e-bbe1-a10bf7c7947d admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ba9b2ff218145d38bf1485a55ef654e, user_id=b1f498cc67d34da5880626afc9eabc51, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.714631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b81069-c92f-4b5e-bbe1-a10bf7c7947d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.726404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 936/1843] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:40 2022] PUT /identity/v3/projects/5ba9b2ff218145d38bf1485a55ef654e/users/b1f498cc67d34da5880626afc9eabc51/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:40.737715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.738001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.738319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.741622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:40.742254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:40.742540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:40.742821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:40.779138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] MFA Rules not processed for user `b1f498cc67d34da5880626afc9eabc51`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:40.780559 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:40.811063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:40.812568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:40.823863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 908/1844] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:40.877219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 937/1845] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:40 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:41.044643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:41.060024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a8a6c4-7946-4152-a0e2-822f88357403 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:41.120413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:41.120981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:41.121290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:41.121564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:41.137321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:41.138804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:41.149507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 909/1846] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:41 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:41.380590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:41.389947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f916a-0690-4b21-a75d-6b278c6188af None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:41.398103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94cbbc1-1e3d-4bb6-8b42-0138c1d94a3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:41.398710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cbbc1-1e3d-4bb6-8b42-0138c1d94a3e service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:41.398984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cbbc1-1e3d-4bb6-8b42-0138c1d94a3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:41.399325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94cbbc1-1e3d-4bb6-8b42-0138c1d94a3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:41.460954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cbbc1-1e3d-4bb6-8b42-0138c1d94a3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:41.462577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94cbbc1-1e3d-4bb6-8b42-0138c1d94a3e service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:41.472429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 938/1847] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:41 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:43.440377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:43.537038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb86eef-12aa-41a3-8c4a-b1a1b515719a service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:43.542454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:43.543250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:43.543540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:43.543882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] PATH_INFO: `/v3/users/49d8b06354dd459ebd7eb8f66b2abdb3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:43.545110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49d8b06354dd459ebd7eb8f66b2abdb3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:43.547117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:43.923521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49d8b06354dd459ebd7eb8f66b2abdb3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.924766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49d8b06354dd459ebd7eb8f66b2abdb3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.936996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49d8b06354dd459ebd7eb8f66b2abdb3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.937685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49d8b06354dd459ebd7eb8f66b2abdb3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:43.944713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49d8b06354dd459ebd7eb8f66b2abdb3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.043186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 910/1848] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:43 2022] DELETE /identity/v3/users/49d8b06354dd459ebd7eb8f66b2abdb3 => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:44.046468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.047614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.048099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.070164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:44.071810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.147195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 939/1849] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:44.157329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.161519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.230224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1578a1e5-d8eb-4aca-8cae-2fa48db40d37 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.234485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4395e-2835-4f20-a7ed-6bed58d6152e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:44.241207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4395e-2835-4f20-a7ed-6bed58d6152e service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.241553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4395e-2835-4f20-a7ed-6bed58d6152e service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.241840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4395e-2835-4f20-a7ed-6bed58d6152e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.263357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4395e-2835-4f20-a7ed-6bed58d6152e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:44.263632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.264807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4395e-2835-4f20-a7ed-6bed58d6152e service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:44.265129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.282078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 911/1850] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:44.282381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1140dfb-2b11-414b-8281-15cbbf7a0c66 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.297318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:44.297668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.297951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.298279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] PATH_INFO: `/v3/users/ef6fa31a87c84df8bb125426ca479572` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.299983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ef6fa31a87c84df8bb125426ca479572)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:44.302295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:44.501788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:44.603041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ef6fa31a87c84df8bb125426ca479572'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.606701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ef6fa31a87c84df8bb125426ca479572'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.606701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ef6fa31a87c84df8bb125426ca479572'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.613981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ef6fa31a87c84df8bb125426ca479572'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.629048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ef6fa31a87c84df8bb125426ca479572'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:44.729492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 940/1851] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:44 2022] DELETE /identity/v3/users/ef6fa31a87c84df8bb125426ca479572 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:44.734017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.734489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.734803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.756677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:44.758066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.840968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 912/1852] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:44.855485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.861373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.901339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:44.906122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:44.908149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:44.924958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48ff153-b301-4753-8fc4-958abea5ba6f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.930355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:44.931289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.931668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.932114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] PATH_INFO: `/v3/projects/27f3464b928a4870984f70699ddf0292` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.933463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f3464b928a4870984f70699ddf0292)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:44.935473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:44.946570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f9dc80-1358-4559-8691-434c95ddeeac admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:44.950896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f8d7c2-6a8e-43f5-9173-49d5a2a41416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:44.951629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d7c2-6a8e-43f5-9173-49d5a2a41416 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.952940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d7c2-6a8e-43f5-9173-49d5a2a41416 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.952940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f8d7c2-6a8e-43f5-9173-49d5a2a41416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.963910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d7c2-6a8e-43f5-9173-49d5a2a41416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:44.965632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f8d7c2-6a8e-43f5-9173-49d5a2a41416 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:44.982982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 941/1853] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:44.983689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cfdd8-433b-4809-80fe-aee092900dd4 None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.984104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cfdd8-433b-4809-80fe-aee092900dd4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.984395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590cfdd8-433b-4809-80fe-aee092900dd4 None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:44.986513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 942/1854] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:31:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:31:44.993273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:44.993587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:44.993829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.021736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:45.023324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:45.116913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 943/1855] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:31:44 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:45.133645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:45.135209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:45.153586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e45e9004-4e08-4885-a3e7-7d087b722bfd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:45.167188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f3464b928a4870984f70699ddf0292', 'request_id': 'req-019b8a2f-1aab-4cce-8496-36ec007cd370'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.174878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f3464b928a4870984f70699ddf0292', 'request_id': 'req-019b8a2f-1aab-4cce-8496-36ec007cd370'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.186603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f3464b928a4870984f70699ddf0292', 'request_id': 'req-019b8a2f-1aab-4cce-8496-36ec007cd370'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.255626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:45.256251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:45.256542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:45.257287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.267104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:45.289051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:45.293419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 944/1856] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:45.347837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] The token cache is being invalidate because project 27f3464b928a4870984f70699ddf0292 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:45.348460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.349251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.352882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 913/1857] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:44 2022] DELETE /identity/v3/projects/27f3464b928a4870984f70699ddf0292 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:45.356476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:45.432651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d9f5ee-5d77-4254-ba3b-0993b4c24bcd service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:45.437789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:45.438444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:45.438730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:45.439165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] PATH_INFO: `/v3/users/b1f498cc67d34da5880626afc9eabc51` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.440832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1f498cc67d34da5880626afc9eabc51)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:45.442944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:45.564544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:45.564544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:45.583171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b8a2f-1aab-4cce-8496-36ec007cd370 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:45.594855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:45.594855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:45.594855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:45.594855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] PATH_INFO: `/v3/projects/0014fc7a4cde4dbbbf33a234378c79dd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:45.596893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0014fc7a4cde4dbbbf33a234378c79dd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:45.596893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:45.828726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1f498cc67d34da5880626afc9eabc51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.829462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1f498cc67d34da5880626afc9eabc51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.830059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1f498cc67d34da5880626afc9eabc51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.833456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0014fc7a4cde4dbbbf33a234378c79dd', 'request_id': 'req-51d08707-bdbd-42fe-a24c-497dc164369f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.838921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1f498cc67d34da5880626afc9eabc51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.840982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0014fc7a4cde4dbbbf33a234378c79dd', 'request_id': 'req-51d08707-bdbd-42fe-a24c-497dc164369f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.847953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0014fc7a4cde4dbbbf33a234378c79dd', 'request_id': 'req-51d08707-bdbd-42fe-a24c-497dc164369f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:45.854486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1f498cc67d34da5880626afc9eabc51'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.070012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] The token cache is being invalidate because project 0014fc7a4cde4dbbbf33a234378c79dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:46.107500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.107500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.107500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 914/1858] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:45 2022] DELETE /identity/v3/projects/0014fc7a4cde4dbbbf33a234378c79dd => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:46.110419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.112368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:46.199164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d08707-bdbd-42fe-a24c-497dc164369f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:46.199164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:46.199164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:46.199164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:46.201173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] PATH_INFO: `/v3/users/03fc47eacc3d4d509d0876512532fdfd` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:46.201173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=03fc47eacc3d4d509d0876512532fdfd)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:46.201173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:46.355983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 945/1859] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:45 2022] DELETE /identity/v3/users/b1f498cc67d34da5880626afc9eabc51 => generated 0 bytes in 1000 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:46.575181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03fc47eacc3d4d509d0876512532fdfd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.575955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03fc47eacc3d4d509d0876512532fdfd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.606966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03fc47eacc3d4d509d0876512532fdfd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.606966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03fc47eacc3d4d509d0876512532fdfd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.606966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03fc47eacc3d4d509d0876512532fdfd'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:46.738186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 915/1860] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:46 2022] DELETE /identity/v3/users/03fc47eacc3d4d509d0876512532fdfd => generated 0 bytes in 618 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:46.882223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:46.882223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:46.967386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb7f034-7b30-4b20-b126-8536a64076a3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.029958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.030514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.030795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.031069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] PATH_INFO: `/v3/projects/5ba9b2ff218145d38bf1485a55ef654e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.032337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ba9b2ff218145d38bf1485a55ef654e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.034527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.158432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ba9b2ff218145d38bf1485a55ef654e', 'request_id': 'req-ae134e22-ebf0-46e5-8bbe-819701a638e0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.166864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ba9b2ff218145d38bf1485a55ef654e', 'request_id': 'req-ae134e22-ebf0-46e5-8bbe-819701a638e0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.316823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ba9b2ff218145d38bf1485a55ef654e', 'request_id': 'req-ae134e22-ebf0-46e5-8bbe-819701a638e0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.324678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.333904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.342806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59883d85-6090-4b7d-a048-25dceac3bf36 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.343018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] The token cache is being invalidate because project 5ba9b2ff218145d38bf1485a55ef654e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:47.345163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.356165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae134e22-ebf0-46e5-8bbe-819701a638e0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.356165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 946/1861] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:46 2022] DELETE /identity/v3/projects/5ba9b2ff218145d38bf1485a55ef654e => generated 0 bytes in 472 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:47.363118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b1868-0bd2-41bd-b65d-ef266e9df23f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.363585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b1868-0bd2-41bd-b65d-ef266e9df23f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.363870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11b1868-0bd2-41bd-b65d-ef266e9df23f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.409433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d11b1868-0bd2-41bd-b65d-ef266e9df23f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:47.409433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d11b1868-0bd2-41bd-b65d-ef266e9df23f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.424002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.425260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.425827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.425827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] PATH_INFO: `/v3/projects/ab02b558fdbe428a92d7fd426d20f8fc` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.426976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab02b558fdbe428a92d7fd426d20f8fc)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.429102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.466057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 947/1862] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:47.466057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.466057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.466057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.482047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:47.483277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.491597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 948/1863] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:47.495730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.505373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00c59a-ee5c-427e-a5b6-fed2093a025d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.510396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.510995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.511280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.511557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.512210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:47.512513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.514147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.526654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 949/1864] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:31:47.530968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.539323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab02b558fdbe428a92d7fd426d20f8fc', 'request_id': 'req-b9808f44-e411-4394-ab2d-02899f900b69'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.541249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f716f723-a4b0-48a0-97c7-a23cde40bbf2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.545321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab02b558fdbe428a92d7fd426d20f8fc', 'request_id': 'req-b9808f44-e411-4394-ab2d-02899f900b69'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.552267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab02b558fdbe428a92d7fd426d20f8fc', 'request_id': 'req-b9808f44-e411-4394-ab2d-02899f900b69'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.602395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.605158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.605158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.605158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.605158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.606105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.629314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] The token cache is being invalidate because project ab02b558fdbe428a92d7fd426d20f8fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:47.629608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.630249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b9808f44-e411-4394-ab2d-02899f900b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:47.631908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 916/1865] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:47 2022] DELETE /identity/v3/projects/ab02b558fdbe428a92d7fd426d20f8fc => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:47.636666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f3a08-c107-437b-a898-d010dcf135d1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.636959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f3a08-c107-437b-a898-d010dcf135d1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.637265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6f3a08-c107-437b-a898-d010dcf135d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.657111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-fe6f3a08-c107-437b-a898-d010dcf135d1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:47.658332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fe6f3a08-c107-437b-a898-d010dcf135d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.667374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 917/1866] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:47.676064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.676426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.676773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.698521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:47.699846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.708895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 918/1867] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:47.712932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.713319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 950/1868] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/projects => generated 387 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:47.715955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.717741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:47.724148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcf3329-41e7-48f3-9cfc-73f426cebd3f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.790953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.791576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.791866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.792197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.792861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:47.793218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.794710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52723d20-16e7-49d9-a06a-18b1b9e65f2b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.794877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.800513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.801321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.801518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.805276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.805276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.805276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.808195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 919/1869] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:31:47.812367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.823101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bbaf08-a2f7-472c-a2d3-8acf8376e0f0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.828146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.834625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.834625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.834625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.834625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.834625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:47.973634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 951/1870] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/users => generated 294 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:47.977946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.988945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908febcc-41dc-44cb-9851-a529e7fa4134 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:47.991454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 920/1871] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/projects => generated 409 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:47.993996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:47.994621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:47.994988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:47.995178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:47.995836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:47.996040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:47.997567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.011977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 952/1872] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:48.013459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.024709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b243c2e6-3649-4a55-8560-5610b7c47624 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.067749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ea438d-0351-4938-96ac-3c871fe164ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.072801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.073604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.073883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.074154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.074774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.076720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.084229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.085038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.085324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.085638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.086235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.087838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.098222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 953/1873] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:48.102605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.112794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf1c513-d634-40f6-abe5-09986bd9e65c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.117823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f914191-681a-49e1-ba9a-ea4555954555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.118487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f914191-681a-49e1-ba9a-ea4555954555 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.118770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f914191-681a-49e1-ba9a-ea4555954555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.119050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f914191-681a-49e1-ba9a-ea4555954555 admin admin] PATH_INFO: `/v3/projects/2aa6f9954eb8424e8a1d9b3fa77cd944/users/1ca90c154b714c1799db7161a9d93ee2/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.134249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f914191-681a-49e1-ba9a-ea4555954555 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aa6f9954eb8424e8a1d9b3fa77cd944, user_id=1ca90c154b714c1799db7161a9d93ee2, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.136953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f914191-681a-49e1-ba9a-ea4555954555 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.281090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 921/1874] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:47 2022] POST /identity/v3/users => generated 305 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:48.287985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.294725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:48.310987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 954/1875] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:48 2022] PUT /identity/v3/projects/2aa6f9954eb8424e8a1d9b3fa77cd944/users/1ca90c154b714c1799db7161a9d93ee2/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:48.311208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4c9e18-1a0a-475d-9eb4-4146b847d1ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.313511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.314004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.314285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.359294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] MFA Rules not processed for user `1ca90c154b714c1799db7161a9d93ee2`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:48.359294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:48.396407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.397703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.398141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.398443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.399510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.402234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.416661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 922/1876] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:48.419509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.433461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4a3553-f9dd-4966-955b-6f78341bb58d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.439555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.439699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 955/1877] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:48 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:48.441080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.441510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.441790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.442445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.444286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.444900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.455160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e78a-54a9-4e8e-96e2-2fcac0b3c5d2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.455988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 923/1878] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:48.460053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.460273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.460888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.461234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.461510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.462173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.464118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.470117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f936fc8-ce35-4715-84d0-29c58ac7257e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.475235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddec7b27-0a34-4452-9d55-705ab1c4cf57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.476086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec7b27-0a34-4452-9d55-705ab1c4cf57 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.476733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec7b27-0a34-4452-9d55-705ab1c4cf57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.477139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddec7b27-0a34-4452-9d55-705ab1c4cf57 admin admin] PATH_INFO: `/v3/projects/b9c2d257fce4491592646792ba64efd7/users/7a8a647aa4b94edaad9d424d9b14c477/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.496967 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec7b27-0a34-4452-9d55-705ab1c4cf57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9c2d257fce4491592646792ba64efd7, user_id=7a8a647aa4b94edaad9d424d9b14c477, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.499718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddec7b27-0a34-4452-9d55-705ab1c4cf57 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.609001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 956/1879] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:48 2022] POST /identity/v3/projects => generated 387 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:48.613931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.619687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 924/1880] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:48 2022] PUT /identity/v3/projects/b9c2d257fce4491592646792ba64efd7/users/7a8a647aa4b94edaad9d424d9b14c477/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:48.632171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.632457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.632735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.664527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] MFA Rules not processed for user `7a8a647aa4b94edaad9d424d9b14c477`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:48.665789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:48.686742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3c1d25-4ad5-4a5b-9506-9573bfe373d6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.731987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 925/1881] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:48 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:48.736508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.744899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.751035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.751035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.751035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.751035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.751035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.772502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:48.800683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791ad15d-a8e0-4c36-af74-ded3861b62a9 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.805542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.806147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.806452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.806724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.807342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.809261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.864295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 957/1882] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:48 2022] POST /identity/v3/users => generated 300 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:48.868230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.879526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20945a18-bc15-4592-922c-93c24b0bc240 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.884444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.885180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.885466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.885744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.886397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.887991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.901067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 958/1883] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:48.904196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:48.914627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f94bc2a-efbf-4ca6-9254-3d3e653facd0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:48.919706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:48.920457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:48.920740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:48.921048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] PATH_INFO: `/v3/projects/5a4a6bfac4504ae785b8cc1317f8c88b/users/04e06881fa94446faf401d62a892d140/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:48.937284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4a6bfac4504ae785b8cc1317f8c88b, user_id=04e06881fa94446faf401d62a892d140, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:48.940495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:48.993310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 926/1884] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:48 2022] POST /identity/v3/projects => generated 407 bytes in 257 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:48.997708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.040825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 959/1885] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:48 2022] PUT /identity/v3/projects/5a4a6bfac4504ae785b8cc1317f8c88b/users/04e06881fa94446faf401d62a892d140/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:49.045114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.062882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805b27ee-09d7-44a3-af34-148f045e8fd2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.119355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831c639-6d4d-4fdc-9f29-4b8ed6a9a1e6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.123950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.124566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.124849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.125055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.125191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.125690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.125808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.125951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.126208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.126856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.127398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.129493 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.138371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 960/1886] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:49.142571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.152369 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efce88c-f586-4f0e-b168-a773f1fa3d6b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.156991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a5880-9b51-4a9e-9219-ecb84227204a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.157673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5880-9b51-4a9e-9219-ecb84227204a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.157953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5880-9b51-4a9e-9219-ecb84227204a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.158226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5880-9b51-4a9e-9219-ecb84227204a admin admin] PATH_INFO: `/v3/projects/5a4a6bfac4504ae785b8cc1317f8c88b/users/04e06881fa94446faf401d62a892d140/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.160353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5880-9b51-4a9e-9219-ecb84227204a admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a4a6bfac4504ae785b8cc1317f8c88b, user_id=04e06881fa94446faf401d62a892d140, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.163829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5880-9b51-4a9e-9219-ecb84227204a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.174195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 961/1887] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:49 2022] PUT /identity/v3/projects/5a4a6bfac4504ae785b8cc1317f8c88b/users/04e06881fa94446faf401d62a892d140/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:49.185498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.185793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.186169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.215363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] MFA Rules not processed for user `04e06881fa94446faf401d62a892d140`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:49.216586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:49.281318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 927/1888] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:48 2022] POST /identity/v3/users => generated 310 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:49.285371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.287003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 962/1889] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:49.298914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a50ee8b-e47b-43c8-98f0-d787a0b7f463 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.299017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.301261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:49.360971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792b0bf-1e22-4023-aa90-2d49a7201e1d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.361661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.362398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.362732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.363261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.363985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.365588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.365724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.366203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.366501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.366772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.376022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.376191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 928/1890] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:49.378117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.380833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.388568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 963/1891] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:49.390958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b23600-4ab5-4672-9d3b-41d1a09f7fb8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.396674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.396674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.407045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.407045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] PATH_INFO: `/v3/projects/f3858372d93b45e4b35a15fa2a8b76d8/users/1ba8733dbd864f74893559e1f08b1f3a/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.412241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3858372d93b45e4b35a15fa2a8b76d8, user_id=1ba8733dbd864f74893559e1f08b1f3a, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.414924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.439352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.459860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a797d-00eb-40d3-9970-7d2c2e942477 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.459860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.459860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.459860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.460832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] PATH_INFO: `/v3/users/1ca90c154b714c1799db7161a9d93ee2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.460832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ca90c154b714c1799db7161a9d93ee2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.461849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.522613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 929/1892] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:49 2022] PUT /identity/v3/projects/f3858372d93b45e4b35a15fa2a8b76d8/users/1ba8733dbd864f74893559e1f08b1f3a/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:49.526617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.602965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9216976-c3b0-4125-b095-d4ebf9f0f6c6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.609034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.609921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.610215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.610491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.611300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.613973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.625484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 930/1893] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:31:49.629670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.639966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a982521-366a-433c-ba87-0e481d107c75 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.645000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972072af-c142-4739-937c-433f88f171a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.645707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972072af-c142-4739-937c-433f88f171a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.645986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972072af-c142-4739-937c-433f88f171a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.646292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972072af-c142-4739-937c-433f88f171a8 admin admin] PATH_INFO: `/v3/projects/f3858372d93b45e4b35a15fa2a8b76d8/users/1ba8733dbd864f74893559e1f08b1f3a/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.648216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972072af-c142-4739-937c-433f88f171a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3858372d93b45e4b35a15fa2a8b76d8, user_id=1ba8733dbd864f74893559e1f08b1f3a, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.650930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972072af-c142-4739-937c-433f88f171a8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.661831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 931/1894] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:49 2022] PUT /identity/v3/projects/f3858372d93b45e4b35a15fa2a8b76d8/users/1ba8733dbd864f74893559e1f08b1f3a/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:49.680550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.680845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.681173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.709747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] MFA Rules not processed for user `1ba8733dbd864f74893559e1f08b1f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:49.711015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:49.711678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ca90c154b714c1799db7161a9d93ee2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:49.712334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ca90c154b714c1799db7161a9d93ee2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:49.712944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ca90c154b714c1799db7161a9d93ee2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:49.720268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ca90c154b714c1799db7161a9d93ee2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:49.733599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b29bd4c6-f215-4b93-ba02-06aa558d1afa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ca90c154b714c1799db7161a9d93ee2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:49.779300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 932/1895] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:49 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:49.791068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.792327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:49.852094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e544d137-72aa-4364-bb2b-d35cc6a912e7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:49.856134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:49.859762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.859762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.859762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.859992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 964/1896] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:49 2022] DELETE /identity/v3/users/1ca90c154b714c1799db7161a9d93ee2 => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:49.863176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:49.863467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:49.863742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:49.867710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:49.869278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:49.879585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 933/1897] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:49 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:31:49.883081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:49.885029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:49.887263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:49.951010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 965/1898] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:49.966113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:49.967615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.028278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c0f8e-ed99-4369-ba95-082a01c61cd0 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:50.079676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:50.079676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.079676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.079676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.089937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:50.091386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:50.103082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 934/1899] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:50.338825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:50.396390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1003a13d-f6a0-4d9a-9c1f-246f03d2c84b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:50.402487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:50.403560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.404059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.404348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.406169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.475249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:50.476093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:50.486944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 966/1900] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:50.583762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:50.647800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:50.649492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.675464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45045810-fc0e-4573-87e1-66eb9153efbe service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:50.688723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:50.689461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.689812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.690366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.693058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.715351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:50.717515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:50.738671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdac893e-9e51-4842-b800-9b2699401adb service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:50.744328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:50.748347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:50.748347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:50.748347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] PATH_INFO: `/v3/projects/2aa6f9954eb8424e8a1d9b3fa77cd944` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:50.748347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aa6f9954eb8424e8a1d9b3fa77cd944)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:50.748347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:50.788520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:50.788956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:50.814386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 935/1901] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:50 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:50.968347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aa6f9954eb8424e8a1d9b3fa77cd944', 'request_id': 'req-3fcdc3a6-157c-4e94-8763-951f7e5229af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:50.973847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aa6f9954eb8424e8a1d9b3fa77cd944', 'request_id': 'req-3fcdc3a6-157c-4e94-8763-951f7e5229af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.166580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aa6f9954eb8424e8a1d9b3fa77cd944', 'request_id': 'req-3fcdc3a6-157c-4e94-8763-951f7e5229af'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.179450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] The token cache is being invalidate because project 2aa6f9954eb8424e8a1d9b3fa77cd944 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:51.233966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.233966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.233966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 967/1902] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:50 2022] DELETE /identity/v3/projects/2aa6f9954eb8424e8a1d9b3fa77cd944 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:51.234872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:51.234872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469ebab5-8063-4e33-8009-582122a64369 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:51.256847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:51.257522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:51.257805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:51.258115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] PATH_INFO: `/v3/users/04e06881fa94446faf401d62a892d140` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:51.259285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e06881fa94446faf401d62a892d140)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:51.261274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:51.551643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e06881fa94446faf401d62a892d140'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.552336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e06881fa94446faf401d62a892d140'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.564243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e06881fa94446faf401d62a892d140'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.565570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e06881fa94446faf401d62a892d140'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.572801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-25c59e48-952a-4a2e-9510-ba89a211eaab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e06881fa94446faf401d62a892d140'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:51.719191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 936/1903] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:31:51 2022] DELETE /identity/v3/users/04e06881fa94446faf401d62a892d140 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:52.316560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:52.318315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:52.332094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fcdc3a6-157c-4e94-8763-951f7e5229af admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:52.391640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:52.392284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:52.392567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:52.393403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] PATH_INFO: `/v3/projects/5a4a6bfac4504ae785b8cc1317f8c88b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:52.394629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a4a6bfac4504ae785b8cc1317f8c88b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:52.396611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:52.414112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:52.545176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a4a6bfac4504ae785b8cc1317f8c88b', 'request_id': 'req-1073592d-bf52-41a3-a2ae-f3f3168c2f06'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:52.552368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a4a6bfac4504ae785b8cc1317f8c88b', 'request_id': 'req-1073592d-bf52-41a3-a2ae-f3f3168c2f06'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:52.680820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a4a6bfac4504ae785b8cc1317f8c88b', 'request_id': 'req-1073592d-bf52-41a3-a2ae-f3f3168c2f06'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:52.693611 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] The token cache is being invalidate because project 5a4a6bfac4504ae785b8cc1317f8c88b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:31:52.693934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:52.694579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1073592d-bf52-41a3-a2ae-f3f3168c2f06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:31:52.696124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 968/1904] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:31:52 2022] DELETE /identity/v3/projects/5a4a6bfac4504ae785b8cc1317f8c88b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:31:52.701280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:52.701570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:52.702891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:52.722664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:52.723873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:52.789499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 937/1905] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:52.797220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:52.797509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:52.797787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:52.823803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:52.825084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:52.834063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 969/1906] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:52.847512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:52.857773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeef3a7-5ec0-4000-9d93-6b0eb2bca2c5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:52.863322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:52.864013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:52.864361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:52.864708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:52.865399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:31:52.865708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:52.867325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:52.880131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 938/1907] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:31:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:31:52.884922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:52.901897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23e245-12d8-45a9-b7e3-27210cf4f8fe None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:52.901897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:52.901897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:52.901897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:52.902192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:52.902192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:52.902998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:52.947363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:53.043368 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 970/1908] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:52 2022] POST /identity/v3/projects => generated 379 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:53.046972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:53.057578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb078d3-b110-4168-9c9b-86a722b76475 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:53.118984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:53.119758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.120171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.120622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.121530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:53.123700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:53.155257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:53.266852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 939/1909] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:53 2022] POST /identity/v3/users => generated 290 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:53.271435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:53.284524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e69445f-0a97-4819-bb6c-34210a9396ef admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:53.289917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:53.290543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.290814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.291087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.291742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:53.293476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:53.304593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 971/1910] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:53.308971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:53.319525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49c8161-0532-415f-9f48-855263b92fd3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:53.324166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2fbdfe-8ca3-4231-baa7-c48183c0f3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:53.324773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fbdfe-8ca3-4231-baa7-c48183c0f3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.325077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fbdfe-8ca3-4231-baa7-c48183c0f3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.325350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2fbdfe-8ca3-4231-baa7-c48183c0f3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.325989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fbdfe-8ca3-4231-baa7-c48183c0f3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:53.327571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2fbdfe-8ca3-4231-baa7-c48183c0f3b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:53.339088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 940/1911] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:53.344782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:53.355763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933039a2-82b5-4090-83dc-1ccec98d4faf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:53.360345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:53.361099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.361383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.362277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] PATH_INFO: `/v3/projects/b350070a98b44506982b1ff59c586958/users/0759bc06f6cb401bb5a4cf084db1f79e/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.378104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b350070a98b44506982b1ff59c586958, user_id=0759bc06f6cb401bb5a4cf084db1f79e, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:53.380872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:53.498351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 972/1912] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:53 2022] PUT /identity/v3/projects/b350070a98b44506982b1ff59c586958/users/0759bc06f6cb401bb5a4cf084db1f79e/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:53.510046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.510336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.510614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.540581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] MFA Rules not processed for user `0759bc06f6cb401bb5a4cf084db1f79e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:53.541858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:53.619998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 941/1913] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:53 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:53.624580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:53.697002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87343fa0-c3ba-4579-944d-4ce25b54cda0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:53.701854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:53.702489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.702773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.703048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.703691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:53.706326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:53.860621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 973/1914] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:53 2022] POST /identity/v3/projects => generated 379 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:53.864729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:53.875357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a05c9-e08c-47bf-85da-efeda657f0ad None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:53.937069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:53.937709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:53.937991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:53.938234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:53.938888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:53.941990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:54.217706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 942/1915] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:53 2022] POST /identity/v3/users => generated 290 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:54.221884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:54.232468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd53e795-522e-4c5d-9a1d-40e5df32a270 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:54.237138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:54.237755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:54.238028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:54.238303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:54.239115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:54.240698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:54.251231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 974/1916] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:54.255881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:54.266078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36358069-4f25-4be3-b4d0-6ff4a93d4a83 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:54.271434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc5fc60-601f-4159-b6e3-9ed40f328018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:54.272045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5fc60-601f-4159-b6e3-9ed40f328018 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:54.272329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5fc60-601f-4159-b6e3-9ed40f328018 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:54.272603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc5fc60-601f-4159-b6e3-9ed40f328018 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:54.273345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5fc60-601f-4159-b6e3-9ed40f328018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:54.274900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc5fc60-601f-4159-b6e3-9ed40f328018 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:54.285710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 943/1917] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:54.289670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:54.300763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2639e3b-157e-48e8-abb3-78316bcac2ad admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:54.306030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:54.306494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:54.306817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:54.307200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] PATH_INFO: `/v3/projects/5fad6fba867547498a9aa089dd3d4883/users/5231852fd923450f823c5d76a9c97007/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:54.329489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5fad6fba867547498a9aa089dd3d4883, user_id=5231852fd923450f823c5d76a9c97007, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:54.332960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:54.437746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 975/1918] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:54 2022] PUT /identity/v3/projects/5fad6fba867547498a9aa089dd3d4883/users/5231852fd923450f823c5d76a9c97007/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:54.449180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:54.449478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:54.449762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:54.482144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] MFA Rules not processed for user `5231852fd923450f823c5d76a9c97007`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:54.486232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:54.515355 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:54.552211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 944/1919] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:54 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:31:54.556609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:54.621990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14a2a9e-4244-401c-8899-96166229f044 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:54.630213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:54.630846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:54.631128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:54.631458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:54.632096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:54.634158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:54.779828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 976/1920] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:31:54 2022] POST /identity/v3/projects => generated 377 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:31:54.780831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:54.792170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001860af-3b72-4f6d-8eeb-7cedcc7bd5ce None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:54.915988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:54.927496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:54.927779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:54.928295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:54.929054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:54.931781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:55.110527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 945/1921] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:31:54 2022] POST /identity/v3/users => generated 295 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:31:55.119146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:55.130748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6586bb-a2ac-4a67-8e45-6be8602357ba admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:55.135441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:55.136059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.136485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.136768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:55.137453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:55.139061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:55.150193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 977/1922] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:55.154373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:55.166308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1daca2-e27e-44af-860a-cb174e32fb60 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:55.171001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:55.171675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.171955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.172267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] PATH_INFO: `/v3/projects/31d95a2c960c4e5382e8bba52f79cee2/users/dcdac8fdde2047629c2db60c3a40a32e/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:55.188307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d95a2c960c4e5382e8bba52f79cee2, user_id=dcdac8fdde2047629c2db60c3a40a32e, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:55.191932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:55.412901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 946/1923] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:55 2022] PUT /identity/v3/projects/31d95a2c960c4e5382e8bba52f79cee2/users/dcdac8fdde2047629c2db60c3a40a32e/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:55.416890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:55.427765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33452fe-1f7d-4733-9f08-10015bcd64f6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:55.490032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108effc6-9ec1-4dcc-83b1-480a5066eb8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:55.495314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108effc6-9ec1-4dcc-83b1-480a5066eb8f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.495710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108effc6-9ec1-4dcc-83b1-480a5066eb8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.495999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108effc6-9ec1-4dcc-83b1-480a5066eb8f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:55.496753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108effc6-9ec1-4dcc-83b1-480a5066eb8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:55.498478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108effc6-9ec1-4dcc-83b1-480a5066eb8f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:55.510075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 978/1924] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:31:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:31:55.514367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:55.526001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f085e3-cbf1-4a49-807c-7643d92a1bf1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:55.530947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:55.531659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.531944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.532216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] PATH_INFO: `/v3/projects/31d95a2c960c4e5382e8bba52f79cee2/users/dcdac8fdde2047629c2db60c3a40a32e/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:55.534919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] RBAC: Authorizing `identity:create_grant(project_id=31d95a2c960c4e5382e8bba52f79cee2, user_id=dcdac8fdde2047629c2db60c3a40a32e, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:55.537727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:55.557298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 947/1925] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:31:55 2022] PUT /identity/v3/projects/31d95a2c960c4e5382e8bba52f79cee2/users/dcdac8fdde2047629c2db60c3a40a32e/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:31:55.571188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85da732-1820-429d-8169-f816f9209de3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.571188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85da732-1820-429d-8169-f816f9209de3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.571188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85da732-1820-429d-8169-f816f9209de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:55.606149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d85da732-1820-429d-8169-f816f9209de3 None None] MFA Rules not processed for user `dcdac8fdde2047629c2db60c3a40a32e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:31:55.607675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d85da732-1820-429d-8169-f816f9209de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:55.694660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 979/1926] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:31:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:31:55.707153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:55.710021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:55.711320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:31:55.776968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915866da-e169-4294-be78-187bf21ea9ef admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:55.780716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:55.781354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.781713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.781983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:55.791365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:55.792821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:55.802955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 948/1927] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:55 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:55.928263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85da732-1820-429d-8169-f816f9209de3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:31:55.938004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85da732-1820-429d-8169-f816f9209de3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:31:55.942297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840bb143-bb87-481e-8d60-0458327368a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:31:55.942919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb143-bb87-481e-8d60-0458327368a6 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:31:55.943201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb143-bb87-481e-8d60-0458327368a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:31:55.943506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840bb143-bb87-481e-8d60-0458327368a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:31:56.005384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb143-bb87-481e-8d60-0458327368a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:31:56.007022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840bb143-bb87-481e-8d60-0458327368a6 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:31:56.019454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 980/1928] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:31:55 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:31:56.337608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:58.400846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:58.572416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:31:59.559019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:00.017710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:02.313245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:02.680932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:04.428491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:04.548430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:05.182959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:05.274391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:05.521067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:06.057957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:06.274780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:06.606027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:06.925942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:07.268550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:07.697342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:07.937184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:08.124369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:08.125874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:08.203054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d09ccda-5150-49c6-9f43-6451ae97da68 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:08.208691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:08.209362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:08.209643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:08.209916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] PATH_INFO: `/v3/users/2f347b5fc9d24e6bb94d51ea1915a601` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:08.211220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f347b5fc9d24e6bb94d51ea1915a601)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:08.213179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:08.534787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f347b5fc9d24e6bb94d51ea1915a601'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:08.536829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f347b5fc9d24e6bb94d51ea1915a601'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:08.550645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f347b5fc9d24e6bb94d51ea1915a601'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:08.551450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f347b5fc9d24e6bb94d51ea1915a601'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:08.559301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f347b5fc9d24e6bb94d51ea1915a601'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:08.693100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 949/1929] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:08 2022] DELETE /identity/v3/users/2f347b5fc9d24e6bb94d51ea1915a601 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:08.696876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:08.697202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:08.697484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:08.718278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:08.719505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:08.801569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 981/1930] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:08.814566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:08.816239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:08.880993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e86ba2-5f80-4604-bd75-17f6b6580354 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:08.884634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:08.885278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:08.885561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:08.885828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:08.895964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:08.897465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:08.907182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 950/1931] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:08.959776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:09.230077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:09.431014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:09.432468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:09.444979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2085ebc-f763-4dcf-8d4d-332ce278f49c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:09.449802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:09.450438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:09.450718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:09.450989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] PATH_INFO: `/v3/projects/297c354a7753488c924e2e030bbdf7d6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:09.452123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] RBAC: Authorizing `identity:delete_project(project_id=297c354a7753488c924e2e030bbdf7d6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:09.454111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:09.577420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '297c354a7753488c924e2e030bbdf7d6', 'request_id': 'req-41361578-d005-406e-9c0a-bad3553f794d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:09.583140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '297c354a7753488c924e2e030bbdf7d6', 'request_id': 'req-41361578-d005-406e-9c0a-bad3553f794d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:09.708111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '297c354a7753488c924e2e030bbdf7d6', 'request_id': 'req-41361578-d005-406e-9c0a-bad3553f794d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:09.720279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] The token cache is being invalidate because project 297c354a7753488c924e2e030bbdf7d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:09.720479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:09.721158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:09.723311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 982/1932] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:09 2022] DELETE /identity/v3/projects/297c354a7753488c924e2e030bbdf7d6 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:09.727680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:09.738512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b15c65a-ebfd-4e48-98ed-0b40aa4ae815 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:09.794852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:09.795487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:09.795767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:09.796040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] PATH_INFO: `/v3/users/135e93b060094c44b6eacedb6dcfcbb5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:09.797190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=135e93b060094c44b6eacedb6dcfcbb5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:09.799124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:10.061819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '135e93b060094c44b6eacedb6dcfcbb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:10.066869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '135e93b060094c44b6eacedb6dcfcbb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:10.066869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:10.086796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '135e93b060094c44b6eacedb6dcfcbb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:10.087654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '135e93b060094c44b6eacedb6dcfcbb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:10.094559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f38f5936-d368-48e0-b431-8f030527acbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '135e93b060094c44b6eacedb6dcfcbb5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:10.175561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 951/1933] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:09 2022] DELETE /identity/v3/users/135e93b060094c44b6eacedb6dcfcbb5 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:10.177103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:10.653375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:10.654813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:10.660679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:10.671374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41361578-d005-406e-9c0a-bad3553f794d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:10.736542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:10.737568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:10.737875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:10.738149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] PATH_INFO: `/v3/projects/f0f3239778914b19a8af3a503cf86665` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:10.739312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0f3239778914b19a8af3a503cf86665)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:10.741338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:10.863313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:10.869650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0f3239778914b19a8af3a503cf86665', 'request_id': 'req-c96607f1-ffdf-4c44-ae3c-269692adc738'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:10.875344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0f3239778914b19a8af3a503cf86665', 'request_id': 'req-c96607f1-ffdf-4c44-ae3c-269692adc738'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:11.005216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0f3239778914b19a8af3a503cf86665', 'request_id': 'req-c96607f1-ffdf-4c44-ae3c-269692adc738'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:11.016762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] The token cache is being invalidate because project f0f3239778914b19a8af3a503cf86665 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:11.017079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:11.017792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c96607f1-ffdf-4c44-ae3c-269692adc738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:11.019219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 983/1934] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:10 2022] DELETE /identity/v3/projects/f0f3239778914b19a8af3a503cf86665 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:11.024415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.024704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.024988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.045150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:11.046373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:11.119086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 952/1935] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:11.126824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:11.130776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.131367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.131647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.154393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:11.155184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:11.155749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:11.165116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 984/1936] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:11.169548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:11.180686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80df848-2927-4cbb-96b4-d97e8bcef87e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:11.186599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:11.191366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.191366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.191366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.191366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:11.191366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:11.191366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:11.204620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 953/1937] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:11.210377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:11.219693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e5923-1024-4d16-8ecd-97767c25c4df None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:11.226085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:11.237760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.237760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.237760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.237760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:11.237760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:11.257074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:11.340742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 985/1938] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:11 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:11.346944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:11.351642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:11.361878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3835be2-f094-4082-aa14-59bd36d6f5b5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:11.439500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:11.440991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.441308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.441584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.442239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:11.444343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:11.531192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 954/1939] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:11 2022] POST /identity/v3/users => generated 283 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:11.535576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:11.545897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77d1363-7305-451a-8228-089019b786fb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:11.550991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:11.553690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.553690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.553690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.553690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:11.554376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:11.564378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 986/1940] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:11.569172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:11.579444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5336ef77-f8ed-4de5-a16f-b84b57ce57ab admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:11.584155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d2cf28-0794-46b8-8537-3db07eeeae0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:11.584796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2cf28-0794-46b8-8537-3db07eeeae0f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.585239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2cf28-0794-46b8-8537-3db07eeeae0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.585458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d2cf28-0794-46b8-8537-3db07eeeae0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.589237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2cf28-0794-46b8-8537-3db07eeeae0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:11.589237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d2cf28-0794-46b8-8537-3db07eeeae0f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:11.598678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 955/1941] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:11.603134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:11.613942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4298232-9f3f-4bca-b8d1-ab0cea086bb5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:11.618823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:11.619572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.619855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.626347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] PATH_INFO: `/v3/projects/d937fe11237b495a9a0f71b94e417168/users/78500856ed97472ebae52c6bb064db0e/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.636405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d937fe11237b495a9a0f71b94e417168, user_id=78500856ed97472ebae52c6bb064db0e, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:11.639296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:11.755585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 987/1942] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:11 2022] PUT /identity/v3/projects/d937fe11237b495a9a0f71b94e417168/users/78500856ed97472ebae52c6bb064db0e/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:11.771448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:11.811683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:11.811683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:11.811683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:11.843120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] MFA Rules not processed for user `78500856ed97472ebae52c6bb064db0e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:11.844431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:11.908984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 956/1943] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:11.959415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.024054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b935ae-c6e2-48a8-891c-345572a5f9b5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.028823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.029480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.029761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.030031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.030645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.032656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.209575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 988/1944] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:11 2022] POST /identity/v3/projects => generated 363 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:12.220308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.232672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e1ade3-0dab-4808-8bdb-61add9ee8d4f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.303817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.304545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.305042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.305515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.306455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.308663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.314137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:12.498771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 957/1945] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:12 2022] POST /identity/v3/users => generated 288 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:12.500181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.515182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb073d69-7481-4aac-9a5d-2206aa3a434e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.532975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.533875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.534799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.535143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.536263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.538134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.555475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 989/1946] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:12.560774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.573516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc69ef8c-f34d-4fb7-b5a6-490125765a52 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.578793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.579556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.579945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.580289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] PATH_INFO: `/v3/projects/3ce3d6d80121453281f5df7de60e6c7f/users/a766dc3458a44a28a6f0cc08b0152cd9/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.598857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce3d6d80121453281f5df7de60e6c7f, user_id=a766dc3458a44a28a6f0cc08b0152cd9, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.601987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.719647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 958/1947] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:12 2022] PUT /identity/v3/projects/3ce3d6d80121453281f5df7de60e6c7f/users/a766dc3458a44a28a6f0cc08b0152cd9/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:12.723981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.740760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdb63eb-f235-4c41-b71c-144ba0a8b274 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.795111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986ef9da-2e02-468b-ae77-0c00283866df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.797472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef9da-2e02-468b-ae77-0c00283866df admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.797472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef9da-2e02-468b-ae77-0c00283866df admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.797472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986ef9da-2e02-468b-ae77-0c00283866df admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.797472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef9da-2e02-468b-ae77-0c00283866df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.798633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986ef9da-2e02-468b-ae77-0c00283866df admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.809046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 990/1948] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:12.813214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.822713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:12.832032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8686581-77a1-4ff9-8452-6cccef46e1e5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:12.832195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:12.832858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.833222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.833499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] PATH_INFO: `/v3/projects/3ce3d6d80121453281f5df7de60e6c7f/users/a766dc3458a44a28a6f0cc08b0152cd9/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.835458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce3d6d80121453281f5df7de60e6c7f, user_id=a766dc3458a44a28a6f0cc08b0152cd9, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:12.838234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:12.849950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 959/1949] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:12 2022] PUT /identity/v3/projects/3ce3d6d80121453281f5df7de60e6c7f/users/a766dc3458a44a28a6f0cc08b0152cd9/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:12.864343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:12.864650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:12.864922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:12.896079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] MFA Rules not processed for user `a766dc3458a44a28a6f0cc08b0152cd9`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:12.897487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:12.967305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 991/1950] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:12.979198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:12.981777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:13.042668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c34f28-36e8-4167-8b19-b22be3d46893 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:13.046718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:13.047343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:13.047628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:13.047909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:13.058243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:13.059721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:13.070391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 960/1951] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:12 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:13.320125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:13.339733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:13.341166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:13.353535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba81a45-934a-4a46-a7cf-c9958906f04d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:13.357158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb06e7f-7d62-46bb-bf42-cafc5d796863 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:13.357817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb06e7f-7d62-46bb-bf42-cafc5d796863 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:13.358119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb06e7f-7d62-46bb-bf42-cafc5d796863 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:13.358392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb06e7f-7d62-46bb-bf42-cafc5d796863 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:13.417311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb06e7f-7d62-46bb-bf42-cafc5d796863 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:13.418741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb06e7f-7d62-46bb-bf42-cafc5d796863 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:13.428657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 992/1952] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:13.434571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:13.436113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:13.514783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8c508-fcac-46e1-9ea6-e4f54f61c78a service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:13.522154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:13.523142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:13.523435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:13.523702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] PATH_INFO: `/v3/users/0759bc06f6cb401bb5a4cf084db1f79e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:13.524872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] RBAC: Authorizing `identity:delete_user(user_id=0759bc06f6cb401bb5a4cf084db1f79e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:13.526894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:13.565853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:13.783731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0759bc06f6cb401bb5a4cf084db1f79e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:13.786254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0759bc06f6cb401bb5a4cf084db1f79e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:13.806345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0759bc06f6cb401bb5a4cf084db1f79e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:13.807208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0759bc06f6cb401bb5a4cf084db1f79e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:13.817886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0759bc06f6cb401bb5a4cf084db1f79e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:13.989398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 961/1953] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:13 2022] DELETE /identity/v3/users/0759bc06f6cb401bb5a4cf084db1f79e => generated 0 bytes in 555 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:13.993879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:13.994298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:13.994431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:14.014374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:14.015589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:14.076043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 993/1954] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:14.091827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:14.093311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:14.153358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-080e3d3c-6ff5-456a-bdcc-d7a8946f6baf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:14.157387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:14.158088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:14.158373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:14.158683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:14.169435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:14.170897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:14.181310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 962/1955] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:14.779965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:14.779965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:14.798196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64c9653-1504-4000-9a44-7fe3984ee044 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:14.803013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:14.804463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:14.805146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:14.805646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] PATH_INFO: `/v3/projects/b350070a98b44506982b1ff59c586958` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:14.806876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b350070a98b44506982b1ff59c586958)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:14.809064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:14.920305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:15.029964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b350070a98b44506982b1ff59c586958', 'request_id': 'req-19db44a6-5452-4f8d-ba5c-640c009d8e62'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.039159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b350070a98b44506982b1ff59c586958', 'request_id': 'req-19db44a6-5452-4f8d-ba5c-640c009d8e62'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.160320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:15.205127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b350070a98b44506982b1ff59c586958', 'request_id': 'req-19db44a6-5452-4f8d-ba5c-640c009d8e62'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.221715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] The token cache is being invalidate because project b350070a98b44506982b1ff59c586958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:15.221715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.221715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.223963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 994/1956] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:14 2022] DELETE /identity/v3/projects/b350070a98b44506982b1ff59c586958 => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:15.228398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:15.238353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3080ed61-5611-4db3-808b-398bc407d4bd service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:15.299165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:15.300477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:15.300763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:15.301066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] PATH_INFO: `/v3/users/5231852fd923450f823c5d76a9c97007` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:15.302351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5231852fd923450f823c5d76a9c97007)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:15.304285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:15.646593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5231852fd923450f823c5d76a9c97007'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.647306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5231852fd923450f823c5d76a9c97007'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.663629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5231852fd923450f823c5d76a9c97007'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.665534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5231852fd923450f823c5d76a9c97007'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.681308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5231852fd923450f823c5d76a9c97007'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:15.773606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 963/1957] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:15 2022] DELETE /identity/v3/users/5231852fd923450f823c5d76a9c97007 => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:16.051106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:16.240201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:16.255604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.255604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:16.279647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19db44a6-5452-4f8d-ba5c-640c009d8e62 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.348727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.349356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.349638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.349915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] PATH_INFO: `/v3/projects/5fad6fba867547498a9aa089dd3d4883` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.351489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5fad6fba867547498a9aa089dd3d4883)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.359402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.563878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5fad6fba867547498a9aa089dd3d4883', 'request_id': 'req-cb9cac46-286b-4662-af25-60e73395a871'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.570466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5fad6fba867547498a9aa089dd3d4883', 'request_id': 'req-cb9cac46-286b-4662-af25-60e73395a871'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.700297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5fad6fba867547498a9aa089dd3d4883', 'request_id': 'req-cb9cac46-286b-4662-af25-60e73395a871'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.714010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] The token cache is being invalidate because project 5fad6fba867547498a9aa089dd3d4883 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:16.714297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.714966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:16.716468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 995/1958] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:16 2022] DELETE /identity/v3/projects/5fad6fba867547498a9aa089dd3d4883 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:16.720339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.722046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:16.735505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a025b702-16ec-4a6e-a6fd-56ba7d43a9e5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.800070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.800989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.801493 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.802038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] PATH_INFO: `/v3/users/dcdac8fdde2047629c2db60c3a40a32e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.803332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dcdac8fdde2047629c2db60c3a40a32e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.805377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:16.957366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:16.958777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:16.971744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9cac46-286b-4662-af25-60e73395a871 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:16.977203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:16.977725 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:16.978006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:16.978281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] PATH_INFO: `/v3/users/7a8a647aa4b94edaad9d424d9b14c477` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:16.979531 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a8a647aa4b94edaad9d424d9b14c477)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:16.981819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.146826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dcdac8fdde2047629c2db60c3a40a32e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.147522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dcdac8fdde2047629c2db60c3a40a32e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.160294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dcdac8fdde2047629c2db60c3a40a32e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.160901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dcdac8fdde2047629c2db60c3a40a32e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.168746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3987aff6-68d4-4f3c-aa48-8da8507b1790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dcdac8fdde2047629c2db60c3a40a32e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.275973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a8a647aa4b94edaad9d424d9b14c477'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.276637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a8a647aa4b94edaad9d424d9b14c477'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.277453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a8a647aa4b94edaad9d424d9b14c477'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.279437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 964/1959] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:16 2022] DELETE /identity/v3/users/dcdac8fdde2047629c2db60c3a40a32e => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:17.285541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a8a647aa4b94edaad9d424d9b14c477'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.299984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a8a647aa4b94edaad9d424d9b14c477'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:17.396821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 996/1960] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:16 2022] DELETE /identity/v3/users/7a8a647aa4b94edaad9d424d9b14c477 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:17.397120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.397120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.397120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.415079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:17.416344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:17.479242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 965/1961] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:17.490009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:17.491474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:17.557412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743c92ff-12b7-4029-8cce-3cb954213f43 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:17.561330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:17.561925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:17.562213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:17.563358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:17.575251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:17.576772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:17.589701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 997/1962] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:17.697567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:18.083239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.084582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:18.102084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e23593-6863-4858-bccd-642c4af4f015 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.108753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.109545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.110294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.110904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] PATH_INFO: `/v3/projects/31d95a2c960c4e5382e8bba52f79cee2` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.112873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31d95a2c960c4e5382e8bba52f79cee2)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.115302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.243425 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.245456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:18.267161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc3183c-cb72-4555-a63c-6189a20ab443 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.275770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.276582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.277132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.277132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] PATH_INFO: `/v3/projects/b9c2d257fce4491592646792ba64efd7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.278650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9c2d257fce4491592646792ba64efd7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.281182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.333434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31d95a2c960c4e5382e8bba52f79cee2', 'request_id': 'req-56bf90a9-2951-4925-9beb-4c496920dd39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.339559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31d95a2c960c4e5382e8bba52f79cee2', 'request_id': 'req-56bf90a9-2951-4925-9beb-4c496920dd39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.345937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31d95a2c960c4e5382e8bba52f79cee2', 'request_id': 'req-56bf90a9-2951-4925-9beb-4c496920dd39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.387851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:18.497557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9c2d257fce4491592646792ba64efd7', 'request_id': 'req-1142207d-5922-4c0f-b857-674c19f42761'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.503427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9c2d257fce4491592646792ba64efd7', 'request_id': 'req-1142207d-5922-4c0f-b857-674c19f42761'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.513088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] The token cache is being invalidate because project 31d95a2c960c4e5382e8bba52f79cee2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:18.513625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.521684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-56bf90a9-2951-4925-9beb-4c496920dd39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.521684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 966/1963] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:18 2022] DELETE /identity/v3/projects/31d95a2c960c4e5382e8bba52f79cee2 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:18.522517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c5d23-776a-43eb-afad-aba3ce1f4fb7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.522807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c5d23-776a-43eb-afad-aba3ce1f4fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.523085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c5d23-776a-43eb-afad-aba3ce1f4fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.544078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-142c5d23-776a-43eb-afad-aba3ce1f4fb7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:18.545454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-142c5d23-776a-43eb-afad-aba3ce1f4fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:18.619276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 967/1964] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:18.627976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.628262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.628588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.633495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9c2d257fce4491592646792ba64efd7', 'request_id': 'req-1142207d-5922-4c0f-b857-674c19f42761'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.651071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:18.652340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:18.653742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] The token cache is being invalidate because project b9c2d257fce4491592646792ba64efd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:18.654267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.655468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.657729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 998/1965] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:18 2022] DELETE /identity/v3/projects/b9c2d257fce4491592646792ba64efd7 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:18.661982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 968/1966] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:18.666089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.666287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.667904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:18.677350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e9d124-deab-40ce-a5de-3920d6411fe2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.681519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142207d-5922-4c0f-b857-674c19f42761 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.682453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.683082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.683358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.683630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.684291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:18.684592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.686510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.686934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.687154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.687441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.687720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] PATH_INFO: `/v3/users/1ba8733dbd864f74893559e1f08b1f3a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.688964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ba8733dbd864f74893559e1f08b1f3a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.691377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.704362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 969/1967] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:32:18.706066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.717427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cac5fa6-30ef-4add-9cb0-1da838512fac admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.722767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.723522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.723798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.724034 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.724680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.726915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.850289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 970/1968] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:18 2022] POST /identity/v3/projects => generated 377 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:18.851468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:18.913678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ba8733dbd864f74893559e1f08b1f3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.914361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ba8733dbd864f74893559e1f08b1f3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.915177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ba8733dbd864f74893559e1f08b1f3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.917758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313972c-d50b-433f-b75a-25888a97ae20 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:18.922704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:18.923368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:18.933510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:18.933510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:18.933735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ba8733dbd864f74893559e1f08b1f3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:18.933838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:18.933838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:18.939181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ba8733dbd864f74893559e1f08b1f3a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:19.016373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 999/1969] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:18 2022] DELETE /identity/v3/users/1ba8733dbd864f74893559e1f08b1f3a => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:19.016583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 971/1970] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:18 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:19.021347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:19.022999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:19.035864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e606fcc-b287-40e5-abc1-056840efea07 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:19.045433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:19.094219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:19.094873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.095155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.095429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:19.096728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:19.098820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:19.112324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 972/1971] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:19.113760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:19.124297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244115ed-8412-4b7d-b391-20f3e7392395 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:19.129537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1d92c9-c122-4ad9-890d-d12b5cfbbff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:19.130151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d92c9-c122-4ad9-890d-d12b5cfbbff4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.130429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d92c9-c122-4ad9-890d-d12b5cfbbff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.130694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1d92c9-c122-4ad9-890d-d12b5cfbbff4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:19.131338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d92c9-c122-4ad9-890d-d12b5cfbbff4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:19.133061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1d92c9-c122-4ad9-890d-d12b5cfbbff4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:19.144183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1000/1972] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:19 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:19.148499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:19.158401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08dbc83-efc3-49a1-945e-9a88b63c693f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:19.165157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:19.165894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.166172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.166450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] PATH_INFO: `/v3/projects/aae75c8750da4a018795d0bbe08d14ba/users/80e46850eb4142159d85085a5f0b763a/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:19.181714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] RBAC: Authorizing `identity:create_grant(project_id=aae75c8750da4a018795d0bbe08d14ba, user_id=80e46850eb4142159d85085a5f0b763a, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:19.184354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:19.337837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 973/1973] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:19 2022] PUT /identity/v3/projects/aae75c8750da4a018795d0bbe08d14ba/users/80e46850eb4142159d85085a5f0b763a/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:19.351146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.351550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.354449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:19.385177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] MFA Rules not processed for user `80e46850eb4142159d85085a5f0b763a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:19.387539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:19.462899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1001/1974] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:19 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:19.474669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:19.476632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:19.534753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0f57fe-712f-4cba-af2c-885de452f13b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:19.538399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec826a66-7a26-4f35-ad29-a83c9fe2dfe3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:19.539005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826a66-7a26-4f35-ad29-a83c9fe2dfe3 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.539287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826a66-7a26-4f35-ad29-a83c9fe2dfe3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.539562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec826a66-7a26-4f35-ad29-a83c9fe2dfe3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:19.548981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826a66-7a26-4f35-ad29-a83c9fe2dfe3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:19.550510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec826a66-7a26-4f35-ad29-a83c9fe2dfe3 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:19.559664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 974/1975] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:19.612752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:19.614928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:19.696223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d087e0f-0966-4f96-8c06-6f17e4235089 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:19.702079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:19.702799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.703086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.703398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] PATH_INFO: `/v3/projects/f3858372d93b45e4b35a15fa2a8b76d8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:19.704787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3858372d93b45e4b35a15fa2a8b76d8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:19.707691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:19.782766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:19.841043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3858372d93b45e4b35a15fa2a8b76d8', 'request_id': 'req-bddb456e-1907-4398-899a-9c8108a465a2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:19.846609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3858372d93b45e4b35a15fa2a8b76d8', 'request_id': 'req-bddb456e-1907-4398-899a-9c8108a465a2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:19.972133 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:19.972942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3858372d93b45e4b35a15fa2a8b76d8', 'request_id': 'req-bddb456e-1907-4398-899a-9c8108a465a2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:19.989447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] The token cache is being invalidate because project f3858372d93b45e4b35a15fa2a8b76d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:19.990366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:19.990570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bddb456e-1907-4398-899a-9c8108a465a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:19.992313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1002/1976] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:19 2022] DELETE /identity/v3/projects/f3858372d93b45e4b35a15fa2a8b76d8 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:19.997822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:19.998111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:19.998392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.020561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:20.021918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:20.087279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 975/1977] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:20.096111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.096403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.096684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.122851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:20.124233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:20.137868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1003/1978] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:20.144027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:20.156324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05dcd55-5fb4-42ed-bbc7-f278941e0afe None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:20.164791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:20.165894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.166721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.167840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.169492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:20.170126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:20.173649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:20.197840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 976/1979] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:32:20.202155 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:20.212831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4b2e0-3374-496b-b644-4989ae3d7e0e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:20.217949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:20.218566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.218849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.219149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.219801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:20.222541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:20.383913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1004/1980] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:20 2022] POST /identity/v3/projects => generated 365 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:20.388487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:20.399135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c8bdf1-b4b1-454b-8ed6-f6eec309c092 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:20.469743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:20.473788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.473788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.473788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.473788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:20.474500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:20.594938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 977/1981] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:20 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:20.600928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:20.612171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf3b38-4f9c-4265-b9c9-d1724a87b44d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:20.616807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:20.617496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.617775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.618092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.618718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:20.620447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:20.632650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1005/1982] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:20.643615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:20.654792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10af5fad-0fe9-4f77-bf45-b90ee6421aa5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:20.661373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f2e59b-93be-4c04-b37f-89b192f3954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:20.662295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2e59b-93be-4c04-b37f-89b192f3954d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.662800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2e59b-93be-4c04-b37f-89b192f3954d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.663505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f2e59b-93be-4c04-b37f-89b192f3954d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.665570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2e59b-93be-4c04-b37f-89b192f3954d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:20.668681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f2e59b-93be-4c04-b37f-89b192f3954d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:20.692927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 978/1983] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:20 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:20.697501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:20.708802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2b4eb9-a076-4a78-b4f7-e53255edbc3f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:20.713517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:20.714211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.714489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.714763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] PATH_INFO: `/v3/projects/8d4a65de76924170bd6f153735e4b689/users/a65b0d2091044863902af775ace0ce03/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.731036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d4a65de76924170bd6f153735e4b689, user_id=a65b0d2091044863902af775ace0ce03, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:20.733733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:20.835083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1006/1984] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:20 2022] PUT /identity/v3/projects/8d4a65de76924170bd6f153735e4b689/users/a65b0d2091044863902af775ace0ce03/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:20.845612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:20.846556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:20.846849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:20.878448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] MFA Rules not processed for user `a65b0d2091044863902af775ace0ce03`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:20.879682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:20.931251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:20.954636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 979/1985] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:20.959986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.038092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca778e8-29b8-4499-9492-2d6dbb166d64 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.042885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.043587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.043851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.044119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.044748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.046668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.058041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:21.137320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1007/1986] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:20 2022] POST /identity/v3/projects => generated 363 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:21.141273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.153776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1f9206-37eb-4334-99ea-6216165d2942 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.212637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.213302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.218864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.218864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.218864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.218864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.239787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:21.307984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 980/1987] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:21 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:21.312075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.323696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d49072f-6b76-4f18-b270-ee8bc0b2e0e2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.328742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.329380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.329660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.329960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.330757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.333234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.345781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1008/1988] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:21.356781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.367434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b649d6d3-c10d-4095-b2d0-71a8d322794d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.372157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.372823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.373132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.373405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] PATH_INFO: `/v3/projects/e5d66ffb90824e5aa01be8b2f57ae9d9/users/1903b212eacb425b85a1750f837d146e/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.388907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d66ffb90824e5aa01be8b2f57ae9d9, user_id=1903b212eacb425b85a1750f837d146e, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.391587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.456022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 981/1989] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:21 2022] PUT /identity/v3/projects/e5d66ffb90824e5aa01be8b2f57ae9d9/users/1903b212eacb425b85a1750f837d146e/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:21.460347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.470791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7bda6e-7c75-49e9-b208-3c85e9b2adf5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.533666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bfec48-90b2-4f97-8619-6f693ed57f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.534567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfec48-90b2-4f97-8619-6f693ed57f65 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.535075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfec48-90b2-4f97-8619-6f693ed57f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.535643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bfec48-90b2-4f97-8619-6f693ed57f65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.537403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:21.537975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfec48-90b2-4f97-8619-6f693ed57f65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.539703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bfec48-90b2-4f97-8619-6f693ed57f65 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.553847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1009/1990] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:21 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:21.555021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.568120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034db325-ea85-45d0-b29f-99fb0d7e738d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.573424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.574129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.574412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.578206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] PATH_INFO: `/v3/projects/e5d66ffb90824e5aa01be8b2f57ae9d9/users/1903b212eacb425b85a1750f837d146e/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.578206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d66ffb90824e5aa01be8b2f57ae9d9, user_id=1903b212eacb425b85a1750f837d146e, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.581649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.592601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 982/1991] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:21 2022] PUT /identity/v3/projects/e5d66ffb90824e5aa01be8b2f57ae9d9/users/1903b212eacb425b85a1750f837d146e/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:21.604184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.604479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.604751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.632371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] MFA Rules not processed for user `1903b212eacb425b85a1750f837d146e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:21.633604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:21.708372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1010/1992] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:21.722624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:21.724654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:21.756179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:21.788166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51a68bd-06e8-41d0-ac4a-0598c35c8cf4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:21.792706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:21.793355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:21.798866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:21.798866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:21.804030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:21.805497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:21.815906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 983/1993] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:22.106876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:22.117405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05565c8f-0c60-43b8-bd30-3aa7d23f23a0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:22.121057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb74378-bcd1-4700-9385-3f9268822872 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:22.121661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74378-bcd1-4700-9385-3f9268822872 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:22.121941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74378-bcd1-4700-9385-3f9268822872 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:22.122212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb74378-bcd1-4700-9385-3f9268822872 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:22.182690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb74378-bcd1-4700-9385-3f9268822872 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:22.184140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb74378-bcd1-4700-9385-3f9268822872 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:22.194694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1011/1994] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:22.233418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:22.662593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:23.319367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:23.977335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:24.988657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:34.077686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=45 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:36.223138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:50.219352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:50.289455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43ec243-b79a-4bb9-963b-fcbb72c1da8b service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:50.294031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:50.294674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:50.294955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:50.295229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] PATH_INFO: `/v3/users/a65b0d2091044863902af775ace0ce03` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:50.296437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=a65b0d2091044863902af775ace0ce03)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:50.298423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:50.503503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a65b0d2091044863902af775ace0ce03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:50.504244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a65b0d2091044863902af775ace0ce03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:50.517066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a65b0d2091044863902af775ace0ce03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:50.517919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a65b0d2091044863902af775ace0ce03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:50.525603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a65b0d2091044863902af775ace0ce03'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:50.643572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 984/1995] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:50 2022] DELETE /identity/v3/users/a65b0d2091044863902af775ace0ce03 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:50.648703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:50.649197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:50.649285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:50.669101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:50.670389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:50.735451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1012/1996] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:50.745797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:50.750720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:50.810374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f30f2b-a6cd-4023-bf7a-ec1fc99c3b6c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:50.822795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:50.822795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:50.822795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:50.822795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:50.825237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:50.826650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:50.836794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 985/1997] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:51.296569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:51.299469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:51.313483 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec8c781-3eb7-44d0-b67c-186574fe5bf1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:51.320584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:51.321286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:51.321568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:51.321894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] PATH_INFO: `/v3/projects/8d4a65de76924170bd6f153735e4b689` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:51.323095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d4a65de76924170bd6f153735e4b689)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:51.325038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:51.457181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d4a65de76924170bd6f153735e4b689', 'request_id': 'req-fa52dc57-0ebb-4893-905b-5b009584809f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.465095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d4a65de76924170bd6f153735e4b689', 'request_id': 'req-fa52dc57-0ebb-4893-905b-5b009584809f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.540783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d4a65de76924170bd6f153735e4b689', 'request_id': 'req-fa52dc57-0ebb-4893-905b-5b009584809f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.554950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] The token cache is being invalidate because project 8d4a65de76924170bd6f153735e4b689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:51.555293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.556104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.557955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1013/1998] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:51 2022] DELETE /identity/v3/projects/8d4a65de76924170bd6f153735e4b689 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:51.561749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:51.571889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c45f9b-ad6b-439b-8666-9509f4c0987d service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:51.591972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:51.632580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:51.633253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:51.633534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:51.633818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] PATH_INFO: `/v3/users/1903b212eacb425b85a1750f837d146e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:51.634936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1903b212eacb425b85a1750f837d146e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:51.636849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:51.783805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1903b212eacb425b85a1750f837d146e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.784467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1903b212eacb425b85a1750f837d146e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.798264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1903b212eacb425b85a1750f837d146e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.798876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1903b212eacb425b85a1750f837d146e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.809271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1903b212eacb425b85a1750f837d146e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:51.884292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 986/1999] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:51 2022] DELETE /identity/v3/users/1903b212eacb425b85a1750f837d146e => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:52.222928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:52.226761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:52.244067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa52dc57-0ebb-4893-905b-5b009584809f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:52.325903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:52.326555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.326844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.327111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] PATH_INFO: `/v3/users/7fb958fb394443288f24b5dcda37ee7b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.328825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb958fb394443288f24b5dcda37ee7b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:52.330442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:52.388579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:52.390141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:52.404530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389fddb6-865c-471f-a873-9e75c5dc9598 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:52.409970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:52.410771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.411091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.411392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] PATH_INFO: `/v3/projects/e5d66ffb90824e5aa01be8b2f57ae9d9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.412607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d66ffb90824e5aa01be8b2f57ae9d9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:52.415729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:52.571715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:52.727951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d66ffb90824e5aa01be8b2f57ae9d9', 'request_id': 'req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.730221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb958fb394443288f24b5dcda37ee7b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.730981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb958fb394443288f24b5dcda37ee7b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.731843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb958fb394443288f24b5dcda37ee7b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.737139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d66ffb90824e5aa01be8b2f57ae9d9', 'request_id': 'req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.739799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb958fb394443288f24b5dcda37ee7b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.745423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d66ffb90824e5aa01be8b2f57ae9d9', 'request_id': 'req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.758362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-191c463b-40ad-4741-8fa0-584ea438466e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb958fb394443288f24b5dcda37ee7b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.866582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] The token cache is being invalidate because project e5d66ffb90824e5aa01be8b2f57ae9d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:52.866816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.867476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c7b4a8cc-d2a4-47d3-b16a-ae26b8f5de31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:52.869003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 987/2000] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:52 2022] DELETE /identity/v3/projects/e5d66ffb90824e5aa01be8b2f57ae9d9 => generated 0 bytes in 481 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:52.873900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c329b8b-ae31-4c78-b771-0e648f14d4dd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.874176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c329b8b-ae31-4c78-b771-0e648f14d4dd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.874446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c329b8b-ae31-4c78-b771-0e648f14d4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.895068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5c329b8b-ae31-4c78-b771-0e648f14d4dd None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:52.896307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5c329b8b-ae31-4c78-b771-0e648f14d4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:52.916656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:52.955938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1014/2001] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:52 2022] DELETE /identity/v3/users/7fb958fb394443288f24b5dcda37ee7b => generated 0 bytes in 731 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:52.958744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.959276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.969186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.969478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 988/2002] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:52.973463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:52.974031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:52.974320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:52.990601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:52.991964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:52.998497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:53.000119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.060443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1015/2003] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:53.065292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 989/2004] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:53.066422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.071234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.073461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.078004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e0384e-f11a-4eba-ba7b-0d6e8a2c632c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.083023 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.083643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.083924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.084207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.084916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:53.086017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.087939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.106874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1016/2005] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:53.111640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.122558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537c6cf8-b7f0-4990-8f38-4b032f25ca57 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.127365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.128104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.128392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.128669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.129443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.131419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.145626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4142c39-92e9-4c65-8d56-5823aec16f08 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.150187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.150809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.151236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.151662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.161181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.162648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.172772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 990/2006] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:53.279061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1017/2007] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:53.280183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.290646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba8c4fd-10e5-432c-8d9d-aec3b6f32382 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.364925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.365583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.365884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.368355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.368355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.368951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.491071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:53.500673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 991/2008] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/users => generated 293 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:53.502149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.513633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d68173f-7c50-42f6-9bf9-586d78acbe25 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.519995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.521069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.521546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.522039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.523823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.526150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.540726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1018/2009] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:53.541989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.552288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180148f7-ad9c-4c23-b282-83bf80e7e513 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.557407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.558031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.558289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.562142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.562142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.562142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.571496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 992/2010] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:53.575895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.586974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b696fae-43e5-4882-87df-f627b95d0fb9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.591889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4616b1cd-febe-4df2-94c0-0c5d13692e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.592550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616b1cd-febe-4df2-94c0-0c5d13692e21 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.592832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616b1cd-febe-4df2-94c0-0c5d13692e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.593140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4616b1cd-febe-4df2-94c0-0c5d13692e21 admin admin] PATH_INFO: `/v3/projects/54616abe027840bc9ffbbf7d033e844a/users/c417ba9e6ec34e57b19f60a3de64d2ff/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.611681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616b1cd-febe-4df2-94c0-0c5d13692e21 admin admin] RBAC: Authorizing `identity:create_grant(project_id=54616abe027840bc9ffbbf7d033e844a, user_id=c417ba9e6ec34e57b19f60a3de64d2ff, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.614385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4616b1cd-febe-4df2-94c0-0c5d13692e21 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.631184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.633924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.660028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc18f6a8-ccff-4ef3-ba82-fb09f507751d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.668719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.668719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.668719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.668719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] PATH_INFO: `/v3/projects/7020dacd068a42158f3dbb5efc28b007` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.673325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7020dacd068a42158f3dbb5efc28b007)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:53.676042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:53.742369 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1019/2011] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:53 2022] PUT /identity/v3/projects/54616abe027840bc9ffbbf7d033e844a/users/c417ba9e6ec34e57b19f60a3de64d2ff/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:53.752385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.752672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.752986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.782309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] MFA Rules not processed for user `c417ba9e6ec34e57b19f60a3de64d2ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:53.783703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:53.805516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:53.851577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1020/2012] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:53.856151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:53.891861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7020dacd068a42158f3dbb5efc28b007', 'request_id': 'req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.900804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7020dacd068a42158f3dbb5efc28b007', 'request_id': 'req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.910122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7020dacd068a42158f3dbb5efc28b007', 'request_id': 'req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:53.940943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c45ec-e8c7-47fd-8b4a-178c6128d046 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:53.997961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:53.998568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:53.998847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:53.999115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:53.999740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.001698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.042564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] The token cache is being invalidate because project 7020dacd068a42158f3dbb5efc28b007 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:54.042847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.043529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.045056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 993/2013] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:53 2022] DELETE /identity/v3/projects/7020dacd068a42158f3dbb5efc28b007 => generated 0 bytes in 414 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:54.048969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.050385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:54.064304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6a1f7-0859-4d5b-b101-2a0a81c525f8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.069166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.070729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.071019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.071293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] PATH_INFO: `/v3/users/f2e941cc771d4f83b94faaf0c78dbbb8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.072444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2e941cc771d4f83b94faaf0c78dbbb8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.074378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.095086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1021/2014] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:53 2022] POST /identity/v3/projects => generated 385 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:54.097046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.098939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:54.164447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd7bad-b518-47be-880e-9b79731a732a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.169245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.169898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.170178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.170452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.171089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.173157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.281803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1022/2015] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/users => generated 293 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:54.282936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.294241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4445ff-c381-4b30-b0e0-7b765d463043 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.300128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.300743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.301043 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.301324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.301445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2e941cc771d4f83b94faaf0c78dbbb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.301992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.302532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2e941cc771d4f83b94faaf0c78dbbb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.304369 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.316621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1023/2016] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:54.318522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2e941cc771d4f83b94faaf0c78dbbb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.319326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2e941cc771d4f83b94faaf0c78dbbb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.320948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.322729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:54.326838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a229bdcf-245f-4a5b-8d44-44c38545ce49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2e941cc771d4f83b94faaf0c78dbbb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:54.336058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c868e7e7-fbbe-4a15-98f4-4d0f1dced674 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.340929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.341578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.341856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.342128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.342768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.344397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.358995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1024/2017] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:54 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:54.360094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.371835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276bfdc7-d0d6-4097-a8a4-1126fee9418f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.376526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.377219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.377499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.377775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] PATH_INFO: `/v3/projects/a33202e54b4f4b298cf599414609eb16/users/2c2db222619b45a6aa346bf176901d50/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.392773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33202e54b4f4b298cf599414609eb16, user_id=2c2db222619b45a6aa346bf176901d50, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.395518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.449168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 994/2018] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:54 2022] DELETE /identity/v3/users/f2e941cc771d4f83b94faaf0c78dbbb8 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:54.518905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1025/2019] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:54 2022] PUT /identity/v3/projects/a33202e54b4f4b298cf599414609eb16/users/2c2db222619b45a6aa346bf176901d50/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:54.531028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.531535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.531666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.564164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] MFA Rules not processed for user `2c2db222619b45a6aa346bf176901d50`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:54.565435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:54.596841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:54.603838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:54.636014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 995/2020] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:54.640587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.723247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5fe68e-7053-49f3-b3aa-4c98f9c31e0e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.731011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.731705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.732012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.732277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.733071 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.735119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:54.857904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1026/2021] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/projects => generated 385 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:54.861741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.872579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a55386-33d5-4f3e-bda4-99f156659069 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:54.925307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:54.928631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:54.941313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:54.941964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:54.945380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:54.945925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:54.946807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:54.950253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.052641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55b7302-00a1-4002-8fcb-392a254df8f4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.058192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.059051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.059367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.059643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] PATH_INFO: `/v3/projects/bfaae4ceaee54d1ba21f172b6a7fc9ac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.060820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfaae4ceaee54d1ba21f172b6a7fc9ac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.062810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.105810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 996/2022] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:54 2022] POST /identity/v3/users => generated 299 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:55.107431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.110749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.126315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c264f945-2fac-472c-a8d4-a924399b0ce8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.142189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.142189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.142189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.142189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] PATH_INFO: `/v3/users/78500856ed97472ebae52c6bb064db0e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.142477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78500856ed97472ebae52c6bb064db0e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.142477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.229231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:55.245303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfaae4ceaee54d1ba21f172b6a7fc9ac', 'request_id': 'req-fe0b92db-81e5-4ef8-b13e-46e64601dd93'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.251008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfaae4ceaee54d1ba21f172b6a7fc9ac', 'request_id': 'req-fe0b92db-81e5-4ef8-b13e-46e64601dd93'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.386295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfaae4ceaee54d1ba21f172b6a7fc9ac', 'request_id': 'req-fe0b92db-81e5-4ef8-b13e-46e64601dd93'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.398343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] The token cache is being invalidate because project bfaae4ceaee54d1ba21f172b6a7fc9ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:55.398630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.399451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.400970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1027/2023] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:54 2022] DELETE /identity/v3/projects/bfaae4ceaee54d1ba21f172b6a7fc9ac => generated 0 bytes in 476 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:55.401883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.403360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.441651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78500856ed97472ebae52c6bb064db0e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.442317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78500856ed97472ebae52c6bb064db0e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.456407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78500856ed97472ebae52c6bb064db0e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.457164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78500856ed97472ebae52c6bb064db0e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.464303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-466bd196-2f44-4e97-9874-3869efad83a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78500856ed97472ebae52c6bb064db0e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:55.468028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe0b92db-81e5-4ef8-b13e-46e64601dd93 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.473087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0707863f-8977-4b1a-9468-db63ca4201de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.473950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707863f-8977-4b1a-9468-db63ca4201de admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.474228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707863f-8977-4b1a-9468-db63ca4201de admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.474548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0707863f-8977-4b1a-9468-db63ca4201de admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.475190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707863f-8977-4b1a-9468-db63ca4201de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.476841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0707863f-8977-4b1a-9468-db63ca4201de admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.487012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1028/2024] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:55.491242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.491242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.491242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.509723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:55.510952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.520179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1029/2025] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:55.520179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.520179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.532722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc0837b-881b-4afe-ae89-254273d9917b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.537349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30c8573-30fd-42fc-b542-941eff4b2d5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.538108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c8573-30fd-42fc-b542-941eff4b2d5b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.538471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c8573-30fd-42fc-b542-941eff4b2d5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.538748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30c8573-30fd-42fc-b542-941eff4b2d5b admin admin] PATH_INFO: `/v3/projects/7d124d74379246af91245cad86b056f5/users/de83ff4dd22d4e2e8a913be13c8a0e2b/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.554714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c8573-30fd-42fc-b542-941eff4b2d5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d124d74379246af91245cad86b056f5, user_id=de83ff4dd22d4e2e8a913be13c8a0e2b, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.557635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30c8573-30fd-42fc-b542-941eff4b2d5b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.575635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 997/2026] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:55 2022] DELETE /identity/v3/users/78500856ed97472ebae52c6bb064db0e => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:55.576965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.577394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.577563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.598163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:55.599633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.632956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:55.643960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1030/2027] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:55 2022] PUT /identity/v3/projects/7d124d74379246af91245cad86b056f5/users/de83ff4dd22d4e2e8a913be13c8a0e2b/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:55.645325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.645613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.645891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.664539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:55.666942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 998/2028] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:55.667876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.670660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.683035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000ecd6d-087e-4317-979a-284ae8c76979 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.754443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.754989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.755272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.755546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.756234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.757714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1031/2029] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:55.758112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.758654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.770085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 999/2030] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:55.771135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.772841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c2100-a942-48c1-a854-baea3aae047f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.772964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:55.778704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.779356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.780655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.780655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.780865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:55.781089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.782741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.796878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1032/2031] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:55.797820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.810940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd65c983-3ae3-4906-bd11-d282185dc1c1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.820647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.821881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.822363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.822598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] PATH_INFO: `/v3/projects/7d124d74379246af91245cad86b056f5/users/de83ff4dd22d4e2e8a913be13c8a0e2b/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.825944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d124d74379246af91245cad86b056f5, user_id=de83ff4dd22d4e2e8a913be13c8a0e2b, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.828953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.831808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:55.845065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1033/2032] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:55 2022] PUT /identity/v3/projects/7d124d74379246af91245cad86b056f5/users/de83ff4dd22d4e2e8a913be13c8a0e2b/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:55.846183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:55.858571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ac940-8cc4-4b12-8799-318739815d2e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.911962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29565f6f-9dc0-4583-827b-77e66808e699 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.912576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29565f6f-9dc0-4583-827b-77e66808e699 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.913451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29565f6f-9dc0-4583-827b-77e66808e699 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.913719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29565f6f-9dc0-4583-827b-77e66808e699 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.923580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58074eb-fbca-48ca-8b19-bdd2ed69d503 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:55.923769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29565f6f-9dc0-4583-827b-77e66808e699 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.925153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29565f6f-9dc0-4583-827b-77e66808e699 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.928367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:55.929536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.929881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.930168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.930800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:55.932756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:55.936036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1000/2033] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:55.937341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:55.937630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:55.937910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:55.968588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] MFA Rules not processed for user `de83ff4dd22d4e2e8a913be13c8a0e2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:55.969906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.033282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1034/2034] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:55 2022] POST /identity/v3/projects => generated 385 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:56.038786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.051936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:56.068383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1001/2035] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:55 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:56.080616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.082197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.108166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53607-ec04-4baa-8bbf-1bd28d197005 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.112873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.113517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.113791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.114063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.114685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.117044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.146104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c996c3b-14fe-402a-8a46-423ca5ec673e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.212321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.212965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.219239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.219239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.219239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.247105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1035/2036] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:56 2022] POST /identity/v3/users => generated 293 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:56.252682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.260318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddac900-c84a-449c-abbd-c6a11a36dcae admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.275198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.275830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.276192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.276475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.277193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.278872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.291741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1036/2037] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:56.295447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.311262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.311736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d7e7a9-f7ee-4826-83ea-47437b9c896f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.313229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.320028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.321169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.321548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.322765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.323528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.325336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.335523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1002/2038] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:56 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:32:56.345802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1037/2039] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:56.346706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.360809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e44a00c-69b9-40a9-92a7-efbfa810c96a service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.372707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511e886-4792-4d86-b0f0-4f2da4eb173d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.373451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e886-4792-4d86-b0f0-4f2da4eb173d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.373736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e886-4792-4d86-b0f0-4f2da4eb173d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.374013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511e886-4792-4d86-b0f0-4f2da4eb173d admin admin] PATH_INFO: `/v3/projects/d31853e070ac4ef0aeb6c17e9fbf9712/users/a6009388b19d4068b8427d3e992acdc1/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.402982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511e886-4792-4d86-b0f0-4f2da4eb173d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d31853e070ac4ef0aeb6c17e9fbf9712, user_id=a6009388b19d4068b8427d3e992acdc1, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.405831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511e886-4792-4d86-b0f0-4f2da4eb173d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.450573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.453281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.462971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:56.469328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1003/2040] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:56 2022] PUT /identity/v3/projects/d31853e070ac4ef0aeb6c17e9fbf9712/users/a6009388b19d4068b8427d3e992acdc1/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:56.488543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.488543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.488543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.497286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e4d607-e4b0-4120-92ba-d0d1f01983e2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.532110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] MFA Rules not processed for user `a6009388b19d4068b8427d3e992acdc1`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:56.533285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.550328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:56.566364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.570577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.570577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.570577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] PATH_INFO: `/v3/projects/d937fe11237b495a9a0f71b94e417168` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.570577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=d937fe11237b495a9a0f71b94e417168)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.571188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.603815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1004/2041] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:56.608426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.620280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922acced-9a86-462e-a7b5-8acf9cb105cc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.625283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.626005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.626305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.626627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.627301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.629249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.787192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd937fe11237b495a9a0f71b94e417168', 'request_id': 'req-4e293d8b-b310-4616-8832-f94b2fec71fb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.793327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd937fe11237b495a9a0f71b94e417168', 'request_id': 'req-4e293d8b-b310-4616-8832-f94b2fec71fb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.806641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1005/2042] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:56 2022] POST /identity/v3/projects => generated 387 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:56.810964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.876717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b25a70-6747-41bb-99db-f9d75e587f21 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.881934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.882547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.882799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.883065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.883662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd937fe11237b495a9a0f71b94e417168', 'request_id': 'req-4e293d8b-b310-4616-8832-f94b2fec71fb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.883796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.886828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:56.898818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] The token cache is being invalidate because project d937fe11237b495a9a0f71b94e417168 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:56.899544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.901336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:56.903167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1038/2043] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:56 2022] DELETE /identity/v3/projects/d937fe11237b495a9a0f71b94e417168 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:56.907252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.908693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.978534 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1006/2044] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:56 2022] POST /identity/v3/users => generated 294 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:32:56.982524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:56.984735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:56.985807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e293d8b-b310-4616-8832-f94b2fec71fb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:56.990730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:56.991573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:56.991936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:56.992212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] PATH_INFO: `/v3/users/a766dc3458a44a28a6f0cc08b0152cd9` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:56.993577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a766dc3458a44a28a6f0cc08b0152cd9)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:56.995563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.000416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfb13f2-4230-4fe2-b60a-07146ed1f2ee admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.005549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.006263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.006644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.007213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.008076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.009812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.025046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1007/2045] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:57.028944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.040277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a2c7ba-adc8-4bba-934d-92411d1b1e81 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.045035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.045665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.045947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.046222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.046858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.048438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.059594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1008/2046] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:32:57.064016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.074149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb52859-0b8f-40aa-8686-474cc0cc3907 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.079274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c633c-92f0-49a3-85df-a8dd5cdf3137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.079946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c633c-92f0-49a3-85df-a8dd5cdf3137 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.080227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c633c-92f0-49a3-85df-a8dd5cdf3137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.080667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c633c-92f0-49a3-85df-a8dd5cdf3137 admin admin] PATH_INFO: `/v3/projects/802fc643c1094af5a13f94a341f5def8/users/46d3266da86e41d9a0e9958ff36166fc/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.096778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c633c-92f0-49a3-85df-a8dd5cdf3137 admin admin] RBAC: Authorizing `identity:create_grant(project_id=802fc643c1094af5a13f94a341f5def8, user_id=46d3266da86e41d9a0e9958ff36166fc, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.099719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c633c-92f0-49a3-85df-a8dd5cdf3137 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.143123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a766dc3458a44a28a6f0cc08b0152cd9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.143812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a766dc3458a44a28a6f0cc08b0152cd9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.144434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a766dc3458a44a28a6f0cc08b0152cd9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.158727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a766dc3458a44a28a6f0cc08b0152cd9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.173042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1009/2047] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:57 2022] PUT /identity/v3/projects/802fc643c1094af5a13f94a341f5def8/users/46d3266da86e41d9a0e9958ff36166fc/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:57.173906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a766dc3458a44a28a6f0cc08b0152cd9'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:57.177219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.184918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.185374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.185717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.207135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.215192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] MFA Rules not processed for user `46d3266da86e41d9a0e9958ff36166fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:57.216766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.254938 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1039/2048] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:32:56 2022] DELETE /identity/v3/users/a766dc3458a44a28a6f0cc08b0152cd9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:32:57.285062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1010/2049] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:57.290529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.291957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.364243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a68de-0834-4e07-abd2-84ae3fa18f86 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.369387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.370011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.370280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.370555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.371313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.373330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.486154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1040/2050] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/projects => generated 387 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:57.490708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.588577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f1f517-6526-443a-b98e-92dcc0caa80e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.672162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.672162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.672162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.672162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.672527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.673364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.765039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1011/2051] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:57 2022] POST /identity/v3/users => generated 300 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:57.768678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.779404 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1b234f-cfbb-4779-bb2d-aaf07b7fecbf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.784232 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.784869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.785188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.785476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.786149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.787858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.803979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1041/2052] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:57.804179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.816058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a91eaf4-589d-4be4-bf09-5207c29890c4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:57.821279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:57.821928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:57.822205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:57.822474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] PATH_INFO: `/v3/projects/e1a97977852d404eada3674347636061/users/6282bbcd6a9d4ba5813cd44e6562f7c5/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:57.836902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a97977852d404eada3674347636061, user_id=6282bbcd6a9d4ba5813cd44e6562f7c5, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:57.839676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:57.919191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1012/2053] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:57 2022] PUT /identity/v3/projects/e1a97977852d404eada3674347636061/users/6282bbcd6a9d4ba5813cd44e6562f7c5/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:57.921610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.923402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:57.930456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:57.951772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:57.955151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4870b-4f44-45f2-baef-50c5f2695d3d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.012809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ed926-5a0f-4577-800f-9051c0977b11 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.018462 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.019077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.022607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.022607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.022607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.022607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.034666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1013/2054] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:58.040959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.048411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.049185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.049466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.049738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] PATH_INFO: `/v3/projects/3ce3d6d80121453281f5df7de60e6c7f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.050935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce3d6d80121453281f5df7de60e6c7f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.052459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6283b829-4568-48ea-b86c-cd9bf2e41736 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.052838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.063407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16da6281-44a3-4659-bf9d-db151773ad42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.063407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da6281-44a3-4659-bf9d-db151773ad42 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.063407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da6281-44a3-4659-bf9d-db151773ad42 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.063407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16da6281-44a3-4659-bf9d-db151773ad42 admin admin] PATH_INFO: `/v3/projects/e1a97977852d404eada3674347636061/users/6282bbcd6a9d4ba5813cd44e6562f7c5/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.064179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da6281-44a3-4659-bf9d-db151773ad42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1a97977852d404eada3674347636061, user_id=6282bbcd6a9d4ba5813cd44e6562f7c5, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.065404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16da6281-44a3-4659-bf9d-db151773ad42 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.088117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1014/2055] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:58 2022] PUT /identity/v3/projects/e1a97977852d404eada3674347636061/users/6282bbcd6a9d4ba5813cd44e6562f7c5/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:58.090668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.090929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.091209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.124492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] MFA Rules not processed for user `6282bbcd6a9d4ba5813cd44e6562f7c5`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:58.125760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.203629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1015/2056] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:58.215043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.216470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.277620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f350c997-fc20-47a7-b655-ceea9592e24e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.298166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce3d6d80121453281f5df7de60e6c7f', 'request_id': 'req-1450f241-1249-4eb8-afe4-58983adbefd7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:58.306671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce3d6d80121453281f5df7de60e6c7f', 'request_id': 'req-1450f241-1249-4eb8-afe4-58983adbefd7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:58.330759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f067a8-128e-40cd-908a-664e3e90639b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.331311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f067a8-128e-40cd-908a-664e3e90639b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.331591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f067a8-128e-40cd-908a-664e3e90639b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.331869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f067a8-128e-40cd-908a-664e3e90639b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.380762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce3d6d80121453281f5df7de60e6c7f', 'request_id': 'req-1450f241-1249-4eb8-afe4-58983adbefd7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:58.389880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:58.393633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] The token cache is being invalidate because project 3ce3d6d80121453281f5df7de60e6c7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:32:58.393986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:58.394644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1450f241-1249-4eb8-afe4-58983adbefd7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:32:58.396193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1042/2057] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:32:57 2022] DELETE /identity/v3/projects/3ce3d6d80121453281f5df7de60e6c7f => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:58.396633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f067a8-128e-40cd-908a-664e3e90639b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.398627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f067a8-128e-40cd-908a-664e3e90639b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.401816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.402162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.402549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.409391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1016/2058] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:32:58.423405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:58.424835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.489437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:58.501375 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1043/2059] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:32:58.504901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.505635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.505808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.527235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:58.528469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:58.536966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1017/2060] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:58.542045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.552056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9e847c-e46b-44a4-a1e7-6377e117199a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.557581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.558211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.558493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.558773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.559440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:32:58.559749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.561484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.578096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1044/2061] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:32:58.579110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.591142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3f635f-ac47-4fa0-a6f6-14862d5b9848 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.596432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.597093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.597375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.597649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.598541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.601263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.700195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1018/2062] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:58.703799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.713821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9fc0e7-9306-4fa1-9470-18acd46474c5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.784512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.785326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.785612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.785884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.786530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.790001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.895019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1045/2063] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:58 2022] POST /identity/v3/users => generated 284 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:58.899304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.914793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f8cd0-f23b-4106-b7b2-826bae804f3c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.920149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.920943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.921441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.921826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.922717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.924962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.939514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1019/2064] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:58.944095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:58.955510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbb614e-5bf2-4950-84a7-fe3ce70e616f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:58.960748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac7b7ca-eba1-4ba9-8422-8beb6fd65aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:58.961523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7b7ca-eba1-4ba9-8422-8beb6fd65aad admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:58.961807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7b7ca-eba1-4ba9-8422-8beb6fd65aad admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:58.962078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac7b7ca-eba1-4ba9-8422-8beb6fd65aad admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:58.962791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7b7ca-eba1-4ba9-8422-8beb6fd65aad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:58.964378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac7b7ca-eba1-4ba9-8422-8beb6fd65aad admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:58.976901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1046/2065] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:59.020402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.031230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a631d9b1-e8d7-4db4-8b48-6d313bb63aba admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.036248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.036922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.037238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.038097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] PATH_INFO: `/v3/projects/23be3e1b849a43ceb80875462e1e3b04/users/2f7ef072ec5443e7823c1c5fb4b6c6ce/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.055324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23be3e1b849a43ceb80875462e1e3b04, user_id=2f7ef072ec5443e7823c1c5fb4b6c6ce, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.058144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.289770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1020/2066] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:59 2022] PUT /identity/v3/projects/23be3e1b849a43ceb80875462e1e3b04/users/2f7ef072ec5443e7823c1c5fb4b6c6ce/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:32:59.301748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.302054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.302327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.330990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] MFA Rules not processed for user `2f7ef072ec5443e7823c1c5fb4b6c6ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:32:59.332203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:32:59.399267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1047/2067] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:32:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:32:59.403993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.466878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7885200c-1491-4bc1-ac84-12404513c3b5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.471539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.472150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.472429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.472708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.473657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.476394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.607485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1021/2068] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:32:59 2022] POST /identity/v3/projects => generated 367 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:59.611592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.622716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7b66b-7fc0-4193-b852-8cae7ef19970 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.680998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.681981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.682265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.682541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.683172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.685444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.880294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1048/2069] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:32:59 2022] POST /identity/v3/users => generated 290 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:32:59.882331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.892836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e486b8-7d1e-499b-8c1c-fc8008547f29 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.897727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.898337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.898620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.898945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.899590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.901212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.904127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:32:59.913525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1022/2070] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:32:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:32:59.918548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:32:59.929390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b7ff50-512a-4cae-8977-5f45d3cf5a08 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:32:59.936002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:32:59.936940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:32:59.937264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:32:59.937549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] PATH_INFO: `/v3/projects/008207ea53c64418971fea8807ce6926/users/0e5f2e2a631d4897af94609f347bd51f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:32:59.954025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] RBAC: Authorizing `identity:create_grant(project_id=008207ea53c64418971fea8807ce6926, user_id=0e5f2e2a631d4897af94609f347bd51f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:32:59.956532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:32:59.997898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:00.092876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1049/2071] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:32:59 2022] PUT /identity/v3/projects/008207ea53c64418971fea8807ce6926/users/0e5f2e2a631d4897af94609f347bd51f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:00.094254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:00.106091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242d29d9-10b4-4e53-a3d3-bc62c0a8d6db admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:00.175944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:00.176589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:00.176871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:00.177179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:00.177822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:00.180237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:00.193040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1023/2072] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:00 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:00.195408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:00.205892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f388f9c-145c-4aac-98b3-52384261a63a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:00.211608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a47f698-b36c-44e5-8461-5691a6956325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:00.212281 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f698-b36c-44e5-8461-5691a6956325 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:00.212597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f698-b36c-44e5-8461-5691a6956325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:00.212904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a47f698-b36c-44e5-8461-5691a6956325 admin admin] PATH_INFO: `/v3/projects/008207ea53c64418971fea8807ce6926/users/0e5f2e2a631d4897af94609f347bd51f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:00.214859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47f698-b36c-44e5-8461-5691a6956325 admin admin] RBAC: Authorizing `identity:create_grant(project_id=008207ea53c64418971fea8807ce6926, user_id=0e5f2e2a631d4897af94609f347bd51f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:00.217648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a47f698-b36c-44e5-8461-5691a6956325 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:00.228283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:00.230283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1050/2073] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:00 2022] PUT /identity/v3/projects/008207ea53c64418971fea8807ce6926/users/0e5f2e2a631d4897af94609f347bd51f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:00.247802 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:00.257473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:00.257995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:00.258394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:00.286594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] MFA Rules not processed for user `0e5f2e2a631d4897af94609f347bd51f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:00.288352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:00.291571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3cae44a-9cd2-4e7f-b0ee-62f9d9a6ad91 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:00.341986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:00.342522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:00.342804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:00.343084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:00.344862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:00.360508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1051/2074] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:00 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:00.373590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:00.375111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:00.387536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aac98ff-3ce1-405a-92b5-4d5979df9efd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:00.391298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:00.391909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:00.392190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:00.392467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:00.402064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:00.403496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:00.409115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:00.410671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe4de77-c813-4be1-85fd-f06e472be30f service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:00.414089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1052/2075] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:00 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:00.422068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1024/2076] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:00 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 194 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:05.058027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:05.092357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:05.148388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa85512c-a3e0-4206-9ad2-42488d520285 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:05.155092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:05.155748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:05.156213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:05.156515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] PATH_INFO: `/v3/users/2f7ef072ec5443e7823c1c5fb4b6c6ce` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:05.158455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f7ef072ec5443e7823c1c5fb4b6c6ce)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:05.161362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:05.470244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f7ef072ec5443e7823c1c5fb4b6c6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.470930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f7ef072ec5443e7823c1c5fb4b6c6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.471536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f7ef072ec5443e7823c1c5fb4b6c6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.480989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f7ef072ec5443e7823c1c5fb4b6c6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.503461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f7ef072ec5443e7823c1c5fb4b6c6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:05.592801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1053/2077] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:05 2022] DELETE /identity/v3/users/2f7ef072ec5443e7823c1c5fb4b6c6ce => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:05.597388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:05.597897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:05.598256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:05.606301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:05.618020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:05.619392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:05.686620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1025/2078] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:05.697307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:05.698727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:05.757994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9b1400-7a9a-4103-a710-986512bf24c1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:05.763402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:05.764020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:05.764296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:05.764704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:05.780663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:05.780663 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:05.787055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1054/2079] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:06.396537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.398053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:06.410489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8666038a-081e-4811-9ae9-1dadbbc4ea1a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.416084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.416777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.417083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.417373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] PATH_INFO: `/v3/projects/23be3e1b849a43ceb80875462e1e3b04` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.418517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23be3e1b849a43ceb80875462e1e3b04)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.420453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:06.546962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23be3e1b849a43ceb80875462e1e3b04', 'request_id': 'req-38cf23ee-0224-40ae-a065-57633f4ddf39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.553246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23be3e1b849a43ceb80875462e1e3b04', 'request_id': 'req-38cf23ee-0224-40ae-a065-57633f4ddf39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.561266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23be3e1b849a43ceb80875462e1e3b04', 'request_id': 'req-38cf23ee-0224-40ae-a065-57633f4ddf39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.723311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] The token cache is being invalidate because project 23be3e1b849a43ceb80875462e1e3b04 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:06.723960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.724559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:06.726799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1026/2080] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:06 2022] DELETE /identity/v3/projects/23be3e1b849a43ceb80875462e1e3b04 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:06.731200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:06.743117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415b293b-d71d-425d-8e4e-c6013e2e0bc2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:06.794634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:06.811300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:06.811921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:06.812201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:06.812480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] PATH_INFO: `/v3/users/0e5f2e2a631d4897af94609f347bd51f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:06.813674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e5f2e2a631d4897af94609f347bd51f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:06.815616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.103594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e5f2e2a631d4897af94609f347bd51f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:07.104331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e5f2e2a631d4897af94609f347bd51f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:07.104959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e5f2e2a631d4897af94609f347bd51f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:07.113685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e5f2e2a631d4897af94609f347bd51f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:07.127951 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e5f2e2a631d4897af94609f347bd51f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:07.238455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1055/2081] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:06 2022] DELETE /identity/v3/users/0e5f2e2a631d4897af94609f347bd51f => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:07.443173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.445941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:07.535026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cf23ee-0224-40ae-a065-57633f4ddf39 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.583069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:07.591208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.591827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.592106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.596856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.596856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:07.659823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.661296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7465a1-b7af-4c22-aad5-7e80281ef159 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:07.673489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1027/2082] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:07 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:07.868577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:07.871680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:07.949362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2928c-6b19-4b17-8e66-083360063a3d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:07.954338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:07.955004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:07.955285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:07.955916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] PATH_INFO: `/v3/projects/008207ea53c64418971fea8807ce6926` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:07.957072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] RBAC: Authorizing `identity:delete_project(project_id=008207ea53c64418971fea8807ce6926)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:07.959008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.062777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '008207ea53c64418971fea8807ce6926', 'request_id': 'req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:08.068389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '008207ea53c64418971fea8807ce6926', 'request_id': 'req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:08.073221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:08.157965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '008207ea53c64418971fea8807ce6926', 'request_id': 'req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:08.172680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] The token cache is being invalidate because project 008207ea53c64418971fea8807ce6926 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:08.173130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:08.173790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3fec7dfc-5073-4aa1-b9e9-ef930fe85382 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:08.175336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1056/2083] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:07 2022] DELETE /identity/v3/projects/008207ea53c64418971fea8807ce6926 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:08.181003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.181535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.181637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.202657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:08.203871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:08.270673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1028/2084] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:08.278526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.278932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.279212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.305675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:08.306924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:08.315915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1057/2085] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:08.321169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.333858 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f97f11b-5790-42e9-bc3a-ff10556f26f8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.339474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.340085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.340368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.340643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.341403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:08.341711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.343349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.356261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1029/2086] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:08.360676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.370904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e73a35-481c-42c7-9289-a15b3faf9ea5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.375544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.376138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.376420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.376691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.377362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.379386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.490308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1058/2087] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:08 2022] POST /identity/v3/projects => generated 369 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:08.495204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.505705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e465c9d3-c394-4f31-a6dc-64c32e4f1206 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.572611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.573268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.573577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.573832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.574477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.576418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.672893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1030/2088] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:08 2022] POST /identity/v3/users => generated 285 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:08.677479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.689291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb61769-9524-4fa7-b9e8-8a575bdeacc0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.694278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.695542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.696035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.696318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.696968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.698620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.709164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1059/2089] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:08.714213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.726186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21fc5c1-3531-4add-a83f-527f9f1b1553 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.732104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4507b61e-3577-410e-9602-f675fe3bba9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.732819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507b61e-3577-410e-9602-f675fe3bba9e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.733288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507b61e-3577-410e-9602-f675fe3bba9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.733590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4507b61e-3577-410e-9602-f675fe3bba9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.734601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507b61e-3577-410e-9602-f675fe3bba9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.736138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4507b61e-3577-410e-9602-f675fe3bba9e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.748947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1031/2090] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:08 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:08.754876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:08.767018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e4b5ad-e3c0-47ae-aae1-57b8abd5449b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:08.771684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:08.772473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.773347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.773624 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] PATH_INFO: `/v3/projects/f404e3d7d53b46ec876c693e236886ac/users/587eba5db9c043c7b4c4ac2152896911/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.788988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f404e3d7d53b46ec876c693e236886ac, user_id=587eba5db9c043c7b4c4ac2152896911, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:08.791840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:08.894902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1060/2091] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:08 2022] PUT /identity/v3/projects/f404e3d7d53b46ec876c693e236886ac/users/587eba5db9c043c7b4c4ac2152896911/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:08.907525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:08.907808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:08.908083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:08.916262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:08.937282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] MFA Rules not processed for user `587eba5db9c043c7b4c4ac2152896911`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:08.938432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:09.005241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1032/2092] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:08 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:09.010441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.071456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713c299d-9fa9-4455-aaa4-302711586623 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.076088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.076812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.077150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.077429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.078051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.079941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.210586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1061/2093] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:09 2022] POST /identity/v3/projects => generated 367 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:09.214982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.226752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43430a8b-4f13-4837-96cc-6ced21292ac5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.284155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.284769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.285083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.285361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.286179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.289715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.415708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1033/2094] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:09 2022] POST /identity/v3/users => generated 290 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:09.419637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.430876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff344fd-55fd-4ecc-8d58-b551725c44be admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.435547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.436163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.436439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.436712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.437394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.439008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.450522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1062/2095] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:09.454680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.465645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd607c6e-9983-47e7-97a8-509ac33550aa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.470572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.471308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.471597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.472005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] PATH_INFO: `/v3/projects/1473f4f9af704b4ba6d45aacbb5e5f67/users/e11b675ed3b84f70b1ae35c826fecae0/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.488944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] RBAC: Authorizing `identity:create_grant(project_id=1473f4f9af704b4ba6d45aacbb5e5f67, user_id=e11b675ed3b84f70b1ae35c826fecae0, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.492594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.574978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1034/2096] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:09 2022] PUT /identity/v3/projects/1473f4f9af704b4ba6d45aacbb5e5f67/users/e11b675ed3b84f70b1ae35c826fecae0/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:09.579060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.589815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0586285-d8b7-4cdd-9cae-8a051be0971a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.645443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113d81f3-5c32-41a2-b6b0-17e11cb1584f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.646054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d81f3-5c32-41a2-b6b0-17e11cb1584f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.646338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d81f3-5c32-41a2-b6b0-17e11cb1584f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.646612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113d81f3-5c32-41a2-b6b0-17e11cb1584f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.647262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d81f3-5c32-41a2-b6b0-17e11cb1584f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.649587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113d81f3-5c32-41a2-b6b0-17e11cb1584f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.651747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:09.659895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1063/2097] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:09 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:09.664380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.675963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab507ee-b479-4133-bd59-a2259c2d80be admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.680981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.681724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.682004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.682283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] PATH_INFO: `/v3/projects/1473f4f9af704b4ba6d45aacbb5e5f67/users/e11b675ed3b84f70b1ae35c826fecae0/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.684213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1473f4f9af704b4ba6d45aacbb5e5f67, user_id=e11b675ed3b84f70b1ae35c826fecae0, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.686911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.697675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1035/2098] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:09 2022] PUT /identity/v3/projects/1473f4f9af704b4ba6d45aacbb5e5f67/users/e11b675ed3b84f70b1ae35c826fecae0/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:09.708009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.708214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.708422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.738917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] MFA Rules not processed for user `e11b675ed3b84f70b1ae35c826fecae0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:09.740181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:09.814122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1064/2099] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:09 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:09.825536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:09.827261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:09.882059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:09.887441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-045600f1-b94a-4518-8103-bd971dbf5460 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:09.891244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:09.891862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:09.892141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:09.892935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:09.903143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:09.904566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:09.916473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1036/2100] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:09 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:10.109983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:10.120006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e3c152-2ede-4023-b07c-79bac4b20760 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:10.124002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:10.124448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:10.124734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:10.125037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:10.189777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:10.191321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:10.202029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1065/2101] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:10 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:10.545173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:10.546604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:10.558569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-365ff432-fefa-4a86-9e12-5cfa7142c666 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:10.562477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:10.563085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:10.563368 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:10.563641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:10.565434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:10.634880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:10.636332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8e76fe-9044-4fec-bc64-bb492e0d607b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:10.646876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1037/2102] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:16.324299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:16.451858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8515a2-e7a7-4972-9451-255ed622c631 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:16.480502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:16.481391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:16.482296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:16.482296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] PATH_INFO: `/v3/users/587eba5db9c043c7b4c4ac2152896911` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:16.489140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=587eba5db9c043c7b4c4ac2152896911)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:16.495064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:16.564809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:16.832302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '587eba5db9c043c7b4c4ac2152896911'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:16.833086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '587eba5db9c043c7b4c4ac2152896911'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:16.833697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '587eba5db9c043c7b4c4ac2152896911'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:16.842069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '587eba5db9c043c7b4c4ac2152896911'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:16.854757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '587eba5db9c043c7b4c4ac2152896911'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:16.969411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1066/2103] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:16 2022] DELETE /identity/v3/users/587eba5db9c043c7b4c4ac2152896911 => generated 0 bytes in 646 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:16.976276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:16.976734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:16.976812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:16.999074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:17.000416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:17.072667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1038/2104] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:17.083044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:17.085307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:17.167599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6180bc6-2537-4376-9cb5-132ce8775b30 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:17.172410 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:17.173400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:17.173568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:17.173795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:17.187526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:17.188970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:17.203132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1067/2105] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:17.686122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:17.690579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:17.704169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9676cdbc-62aa-475e-bd78-640a49e1193e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:17.710883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:17.711745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:17.712228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:17.712584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] PATH_INFO: `/v3/projects/f404e3d7d53b46ec876c693e236886ac` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:17.713989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f404e3d7d53b46ec876c693e236886ac)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:17.715878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:17.915636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f404e3d7d53b46ec876c693e236886ac', 'request_id': 'req-69a6b714-a32e-4d74-9516-fb66497362e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:17.921643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f404e3d7d53b46ec876c693e236886ac', 'request_id': 'req-69a6b714-a32e-4d74-9516-fb66497362e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:17.929759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f404e3d7d53b46ec876c693e236886ac', 'request_id': 'req-69a6b714-a32e-4d74-9516-fb66497362e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.077729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] The token cache is being invalidate because project f404e3d7d53b46ec876c693e236886ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:18.078325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.078674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.080190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1039/2106] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:17 2022] DELETE /identity/v3/projects/f404e3d7d53b46ec876c693e236886ac => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:18.089342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:18.100858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0345ead1-0bbc-424e-87e7-c8a6a0a23b2c service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:18.217396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:18.218845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.219444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.219959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] PATH_INFO: `/v3/users/e11b675ed3b84f70b1ae35c826fecae0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.221979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e11b675ed3b84f70b1ae35c826fecae0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:18.227262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:18.356717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:18.358258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:18.373832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:18.435164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a6b714-a32e-4d74-9516-fb66497362e3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:18.438881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:18.439588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:18.439788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:18.440062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:18.441917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:18.497096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:18.499574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e11b675ed3b84f70b1ae35c826fecae0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.500552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e11b675ed3b84f70b1ae35c826fecae0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.501340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e11b675ed3b84f70b1ae35c826fecae0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.511200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e11b675ed3b84f70b1ae35c826fecae0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.526159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e11b675ed3b84f70b1ae35c826fecae0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:18.541782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:18.543303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:18.552587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1040/2107] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:18 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:18.599660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1068/2108] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:18 2022] DELETE /identity/v3/users/e11b675ed3b84f70b1ae35c826fecae0 => generated 0 bytes in 511 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:19.275297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.276840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:19.292563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e913429-418c-4d11-8e1a-6a477f4e5b12 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.339809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.339809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:19.391941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.393540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.394041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.394647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] PATH_INFO: `/v3/users/80e46850eb4142159d85085a5f0b763a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.397980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=80e46850eb4142159d85085a5f0b763a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.401389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.433463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a81ca9-bca1-44d8-865a-f888367c23b2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.438885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.439609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.439975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.440319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] PATH_INFO: `/v3/projects/1473f4f9af704b4ba6d45aacbb5e5f67` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.442625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1473f4f9af704b4ba6d45aacbb5e5f67)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.444472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:19.749387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80e46850eb4142159d85085a5f0b763a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.750057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80e46850eb4142159d85085a5f0b763a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.751388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1473f4f9af704b4ba6d45aacbb5e5f67', 'request_id': 'req-341359be-27e8-46df-af60-a2b5fcd31ba5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.758318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1473f4f9af704b4ba6d45aacbb5e5f67', 'request_id': 'req-341359be-27e8-46df-af60-a2b5fcd31ba5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.763315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:19.764766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80e46850eb4142159d85085a5f0b763a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.765442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80e46850eb4142159d85085a5f0b763a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.774291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-75ed3433-a971-40dd-9da8-364868de4f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80e46850eb4142159d85085a5f0b763a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.871359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1473f4f9af704b4ba6d45aacbb5e5f67', 'request_id': 'req-341359be-27e8-46df-af60-a2b5fcd31ba5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.884190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] The token cache is being invalidate because project 1473f4f9af704b4ba6d45aacbb5e5f67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:19.884482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.885198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:19.886733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1069/2109] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:19 2022] DELETE /identity/v3/projects/1473f4f9af704b4ba6d45aacbb5e5f67 => generated 0 bytes in 551 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:19.890079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:19.891461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:19.948324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:19.962548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341359be-27e8-46df-af60-a2b5fcd31ba5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:19.967482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:19.968080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:19.968366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:19.968714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] PATH_INFO: `/v3/users/c417ba9e6ec34e57b19f60a3de64d2ff` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:19.973694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c417ba9e6ec34e57b19f60a3de64d2ff)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:19.973694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:20.030927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1041/2110] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:19 2022] DELETE /identity/v3/users/80e46850eb4142159d85085a5f0b763a => generated 0 bytes in 756 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:20.034298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1b29-7f2b-45bc-959e-ad40be2acde7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.034298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1b29-7f2b-45bc-959e-ad40be2acde7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.034298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810f1b29-7f2b-45bc-959e-ad40be2acde7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.052388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-810f1b29-7f2b-45bc-959e-ad40be2acde7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:20.054153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-810f1b29-7f2b-45bc-959e-ad40be2acde7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.114252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1042/2111] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:20.115622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ecd80-629b-4e30-ae54-5ea4d70f3c36 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.116427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ecd80-629b-4e30-ae54-5ea4d70f3c36 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.116724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41ecd80-629b-4e30-ae54-5ea4d70f3c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.137181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f41ecd80-629b-4e30-ae54-5ea4d70f3c36 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:20.138506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f41ecd80-629b-4e30-ae54-5ea4d70f3c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.152386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1043/2112] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:20.153746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.154035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.154320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.179131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:20.180345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.189029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1044/2113] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:20.189848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:20.191135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.254562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6225aae-df83-4557-81a0-183f0d69f4fd None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:20.258378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:20.258993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.259274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.270472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.270472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:20.270472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:20.279671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1045/2114] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:20.280523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:20.291204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26da55-516e-4996-a775-bc54257496fe service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:20.297192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:20.297873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.298176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.298452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.299414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:20.299730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:20.301567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:20.316961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1046/2115] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:20.322744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:20.331177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c417ba9e6ec34e57b19f60a3de64d2ff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:20.332660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c417ba9e6ec34e57b19f60a3de64d2ff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:20.333704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c417ba9e6ec34e57b19f60a3de64d2ff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:20.338866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3c1ee9-6257-4286-b73d-7534e852d01c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:20.348184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:20.349235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.349704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.351021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.352420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:20.353333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c417ba9e6ec34e57b19f60a3de64d2ff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:20.359231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:20.396037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4c32f3ce-c2d3-4080-a2a7-35012350bce0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c417ba9e6ec34e57b19f60a3de64d2ff'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:20.540170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1070/2116] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:19 2022] DELETE /identity/v3/users/c417ba9e6ec34e57b19f60a3de64d2ff => generated 0 bytes in 653 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:20.546915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1047/2117] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:20 2022] POST /identity/v3/projects => generated 375 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:20.547161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.547161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.547161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.547913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:20.559980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.611609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:20.612863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.661435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:20.741136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea799288-b87f-4f53-a5df-1979330f63d5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:20.761245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1071/2118] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:20.771556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:20.772828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.773189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.773470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.774112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:20.776048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:20.782403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:20.784141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:20.920483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1048/2119] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:20 2022] POST /identity/v3/users => generated 288 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:20.926381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:20.946175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8303eb82-3d31-4801-bdb1-410b11f69c66 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:20.953636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d5e7d8-e3df-4f31-a9df-3cfc1dd86ffe admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:20.958324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:20.959440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.960441 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.960854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.967442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:20.968631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:20.969751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:20.974133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:20.974133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:20.977130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:20.990031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:20.990031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.000569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1049/2120] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:20 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:21.010423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.017952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1072/2121] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:21.026940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96caba0-29c1-4576-8125-c67419af60e2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.039452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.039452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.039452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.039452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.039908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.039908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.054768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.056463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:21.058719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1050/2122] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:21 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:21.062663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.073565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e736874b-4b3d-4f12-97f8-20ebb6518747 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.074501 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee21c87-d103-4a8e-99ec-d8577b4f58e7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.079084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.079737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.080015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.080341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] PATH_INFO: `/v3/projects/aae75c8750da4a018795d0bbe08d14ba` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.081802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aae75c8750da4a018795d0bbe08d14ba)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.085246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.086016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f71629-c88f-4460-9439-a974585f5715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.086932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f71629-c88f-4460-9439-a974585f5715 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.087218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f71629-c88f-4460-9439-a974585f5715 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.087502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f71629-c88f-4460-9439-a974585f5715 admin admin] PATH_INFO: `/v3/projects/53e9df6b688945d09b4d23f3f812d33b/users/76307b04d59a4db4bcb65fb5288ed07b/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.129561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:21.133283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f71629-c88f-4460-9439-a974585f5715 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53e9df6b688945d09b4d23f3f812d33b, user_id=76307b04d59a4db4bcb65fb5288ed07b, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.138145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f71629-c88f-4460-9439-a974585f5715 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.277478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1051/2123] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:21 2022] PUT /identity/v3/projects/53e9df6b688945d09b4d23f3f812d33b/users/76307b04d59a4db4bcb65fb5288ed07b/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:21.288192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.289025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.289025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.310110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aae75c8750da4a018795d0bbe08d14ba', 'request_id': 'req-14bc4ae7-129b-4c09-94ba-74413ae7a789'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:21.316203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aae75c8750da4a018795d0bbe08d14ba', 'request_id': 'req-14bc4ae7-129b-4c09-94ba-74413ae7a789'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:21.340520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] MFA Rules not processed for user `76307b04d59a4db4bcb65fb5288ed07b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:21.341353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:21.438994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1052/2124] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:21.443538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.454867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aae75c8750da4a018795d0bbe08d14ba', 'request_id': 'req-14bc4ae7-129b-4c09-94ba-74413ae7a789'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:21.472762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] The token cache is being invalidate because project aae75c8750da4a018795d0bbe08d14ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:21.473396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:21.475215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-14bc4ae7-129b-4c09-94ba-74413ae7a789 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:21.477534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1073/2125] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:21 2022] DELETE /identity/v3/projects/aae75c8750da4a018795d0bbe08d14ba => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:21.482446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e57244-9b2f-4817-8b3e-0e58292db9d2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.484012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e57244-9b2f-4817-8b3e-0e58292db9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.484506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e57244-9b2f-4817-8b3e-0e58292db9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.525754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-32e57244-9b2f-4817-8b3e-0e58292db9d2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:21.527280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-32e57244-9b2f-4817-8b3e-0e58292db9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:21.549939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25977c5a-367a-4209-a4a7-a26fe82fdd3f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.563933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.564634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.564928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.565236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.565877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.567935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.611324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1074/2126] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:21.620273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.633367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.633367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.655536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:21.656816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:21.666988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1075/2127] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:21.671477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.681652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3bd773-326e-49f8-82a4-7921ff52e012 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.687038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.687657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.687968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.688623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.688897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:21.689233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.691009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.709209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1076/2128] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:21.711161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.723044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1053/2129] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:21 2022] POST /identity/v3/projects => generated 375 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:21.723978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072455ef-5c20-4447-8909-378d6c5484b5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.724374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.727117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:21.809567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae08b1f-e13c-4307-8cfb-8163b19a6d37 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:21.818631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.820610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.821389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.821911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] PATH_INFO: `/v3/projects/54616abe027840bc9ffbbf7d033e844a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.822597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:21.823334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:21.823722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:21.824109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:21.824439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54616abe027840bc9ffbbf7d033e844a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.824769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:21.826753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.826893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:21.980235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1077/2130] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:21 2022] POST /identity/v3/projects => generated 369 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:21.980235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:21.980235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:21.989607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54616abe027840bc9ffbbf7d033e844a', 'request_id': 'req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:21.995918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54616abe027840bc9ffbbf7d033e844a', 'request_id': 'req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.002779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54616abe027840bc9ffbbf7d033e844a', 'request_id': 'req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.045573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2080833-f2ca-48d8-bb6c-afd46ac3dc1e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.050260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.050861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.051136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.051403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.052062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.054854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.105178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] The token cache is being invalidate because project 54616abe027840bc9ffbbf7d033e844a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:22.105472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.106124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.107654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1054/2131] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:21 2022] DELETE /identity/v3/projects/54616abe027840bc9ffbbf7d033e844a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:22.108464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.109803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.143668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1078/2132] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:21 2022] POST /identity/v3/users => generated 288 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:22.145220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.147146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.176028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8635e9c2-d437-4ce5-8c89-b066dbf2a045 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.185489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.186138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.186419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.186720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] PATH_INFO: `/v3/users/2c2db222619b45a6aa346bf176901d50` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.187914 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c2db222619b45a6aa346bf176901d50)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.189944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.217531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b179a0-a4d5-40f8-8d9e-c19a394e17f1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.223287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.223889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.224167 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.224434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.225126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.230949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.248833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:22.344131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1055/2133] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:22 2022] POST /identity/v3/users => generated 285 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:22.344131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.344847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.360765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945964c6-4e85-41c0-88a5-09a9a63a4277 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.372752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.372752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.372752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.372752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.373145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.373145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.389369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1056/2134] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:22.390334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.405128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf686cb8-a9c2-452f-8f88-11a71a44ba91 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.415174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c2db222619b45a6aa346bf176901d50'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.415174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c2db222619b45a6aa346bf176901d50'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.415174 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c2db222619b45a6aa346bf176901d50'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.418582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.419287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.419728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.419914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.420589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.422312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.423227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c2db222619b45a6aa346bf176901d50'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.438187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1057/2135] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:22.438986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.440311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c2db222619b45a6aa346bf176901d50'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:22.440885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.461704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b39c43-53da-43c8-83f2-030df96ef3b1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.461704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.461704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.461704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.462851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.463648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.465480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.478710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1058/2136] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:22.479820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.482745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.500992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11178bfc-3222-4d83-b8c3-178a00a74d72 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.509544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.509544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.509544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.509544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.510181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.512208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.524500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1059/2137] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:22.525556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.531363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1079/2138] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:22 2022] DELETE /identity/v3/users/2c2db222619b45a6aa346bf176901d50 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:22.532621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.536670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a66c7f-fdde-4a25-8bad-c2d07e3df4da admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.608670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:22.648135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86422e5-59c9-49f2-91d6-a9b7bca98033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.648843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86422e5-59c9-49f2-91d6-a9b7bca98033 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.649223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86422e5-59c9-49f2-91d6-a9b7bca98033 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.649792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86422e5-59c9-49f2-91d6-a9b7bca98033 admin admin] PATH_INFO: `/v3/projects/344e1abfbee4401c95aad4330eedd5f0/users/5a03bfcbbca74a738bed8d0b1cca0854/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.658336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5c1a09-a75c-4a3c-8f0a-2f21f04f3bd9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.664770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6c0a08-bc3a-4aa4-b62c-c8181fc58939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.665719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0a08-bc3a-4aa4-b62c-c8181fc58939 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.666098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0a08-bc3a-4aa4-b62c-c8181fc58939 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.666382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6c0a08-bc3a-4aa4-b62c-c8181fc58939 admin admin] PATH_INFO: `/v3/projects/c5a8f4534c364a8fb272bf37487a41e6/users/423fdd722f384754a0821ac491ae7856/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.671146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86422e5-59c9-49f2-91d6-a9b7bca98033 admin admin] RBAC: Authorizing `identity:create_grant(project_id=344e1abfbee4401c95aad4330eedd5f0, user_id=5a03bfcbbca74a738bed8d0b1cca0854, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.674312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86422e5-59c9-49f2-91d6-a9b7bca98033 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.685137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0a08-bc3a-4aa4-b62c-c8181fc58939 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5a8f4534c364a8fb272bf37487a41e6, user_id=423fdd722f384754a0821ac491ae7856, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.687974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6c0a08-bc3a-4aa4-b62c-c8181fc58939 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:22.741637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1060/2139] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:22 2022] PUT /identity/v3/projects/344e1abfbee4401c95aad4330eedd5f0/users/5a03bfcbbca74a738bed8d0b1cca0854/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:22.752582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.752582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.752582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.784286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] MFA Rules not processed for user `5a03bfcbbca74a738bed8d0b1cca0854`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:22.785722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.792060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1080/2140] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:22 2022] PUT /identity/v3/projects/c5a8f4534c364a8fb272bf37487a41e6/users/423fdd722f384754a0821ac491ae7856/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:22.802689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.803701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.804128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.840878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] MFA Rules not processed for user `423fdd722f384754a0821ac491ae7856`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:22.842787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.874732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1061/2141] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:22.879400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.901772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:INSERT=5,keystone:SELECT=88,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:22.925119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1081/2142] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:22 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:22.938761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:22.940296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:22.962285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fc00e-86b8-4e2a-88d6-95ca4647f2a3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:22.967174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:22.967791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:22.968072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:22.968344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:22.968973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:22.970922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.029171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9877353f-c99b-4b51-a8bb-8f4f958c3160 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.032915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.033554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.033834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.034106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.045313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.048852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1062/2143] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:22 2022] POST /identity/v3/projects => generated 377 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:23.049037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.052610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.058723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1082/2144] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:22 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:23.158827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e187427c-2c75-4d3c-9247-d4701bc58585 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.170147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.171278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.171943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.172399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.173596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.176505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.223613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.225906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:23.248080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98c420-42fb-4ed9-be72-d09e2eae934f service glance] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.253438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.254117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.254402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.254678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] PATH_INFO: `/v3/projects/a33202e54b4f4b298cf599414609eb16` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.256903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33202e54b4f4b298cf599414609eb16)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.260747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.293177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1063/2145] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:23 2022] POST /identity/v3/users => generated 295 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:23.297677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.319593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c436257f-9fcc-4879-946c-bd0054667f96 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.324151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.324863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.325241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.325579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.326887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.328714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.341045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1064/2146] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:23.346052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.362369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fc46ff-bcd3-4014-a001-8d75eeabb8bf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.371204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.372302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.373471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.373905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] PATH_INFO: `/v3/projects/b0670ba65aa94f85bd9c7bfd6220f715/users/3ecdf9deaa1c448bb16f0db36347cdc4/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.404282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33202e54b4f4b298cf599414609eb16', 'request_id': 'req-d4f530eb-5e01-4b6d-9301-133a4300d491'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.409171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0670ba65aa94f85bd9c7bfd6220f715, user_id=3ecdf9deaa1c448bb16f0db36347cdc4, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.414250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33202e54b4f4b298cf599414609eb16', 'request_id': 'req-d4f530eb-5e01-4b6d-9301-133a4300d491'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.416119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.511495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33202e54b4f4b298cf599414609eb16', 'request_id': 'req-d4f530eb-5e01-4b6d-9301-133a4300d491'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.521401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1065/2147] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:23 2022] PUT /identity/v3/projects/b0670ba65aa94f85bd9c7bfd6220f715/users/3ecdf9deaa1c448bb16f0db36347cdc4/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:23.524764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.543450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] The token cache is being invalidate because project a33202e54b4f4b298cf599414609eb16 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:23.546614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.546614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.546614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1083/2148] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:23 2022] DELETE /identity/v3/projects/a33202e54b4f4b298cf599414609eb16 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:23.550137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.569618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:23.617819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4eec3f-29db-4f30-ab7e-f5138aca68be admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.625843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.627047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.627330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.627608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.628586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.630361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.636934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f530eb-5e01-4b6d-9301-133a4300d491 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.642018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.642830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.643112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.643383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] PATH_INFO: `/v3/users/de83ff4dd22d4e2e8a913be13c8a0e2b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.647982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=de83ff4dd22d4e2e8a913be13c8a0e2b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.648136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1066/2149] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:23 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:23.648235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.649692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:23.652403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:23.669972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a88c9c9c-a920-44db-9cc0-6a9a2e2b01b1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:23.679613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea30130-9ea9-4bb9-8da8-ea1fe78f130c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:23.680358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea30130-9ea9-4bb9-8da8-ea1fe78f130c admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.680706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea30130-9ea9-4bb9-8da8-ea1fe78f130c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.680984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea30130-9ea9-4bb9-8da8-ea1fe78f130c admin admin] PATH_INFO: `/v3/projects/b0670ba65aa94f85bd9c7bfd6220f715/users/3ecdf9deaa1c448bb16f0db36347cdc4/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.683331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea30130-9ea9-4bb9-8da8-ea1fe78f130c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0670ba65aa94f85bd9c7bfd6220f715, user_id=3ecdf9deaa1c448bb16f0db36347cdc4, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:23.685883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea30130-9ea9-4bb9-8da8-ea1fe78f130c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:23.700335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1067/2150] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:23 2022] PUT /identity/v3/projects/b0670ba65aa94f85bd9c7bfd6220f715/users/3ecdf9deaa1c448bb16f0db36347cdc4/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:23.722399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:23.722784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:23.723066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:23.767093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] MFA Rules not processed for user `3ecdf9deaa1c448bb16f0db36347cdc4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:23.768481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:23.769195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:23.841089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1068/2151] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:23 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:23.929952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de83ff4dd22d4e2e8a913be13c8a0e2b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.929952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de83ff4dd22d4e2e8a913be13c8a0e2b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.929952 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de83ff4dd22d4e2e8a913be13c8a0e2b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.946367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de83ff4dd22d4e2e8a913be13c8a0e2b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:23.969971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de83ff4dd22d4e2e8a913be13c8a0e2b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:24.016212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:24.017786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:24.105651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6707b505-18eb-4201-aec3-0f07a08c08cd None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:24.110475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:24.111104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:24.111381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:24.111648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:24.113625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:24.146195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:24.192977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:24.192977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:24.204845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1069/2152] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:24.362851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1084/2153] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:23 2022] DELETE /identity/v3/users/de83ff4dd22d4e2e8a913be13c8a0e2b => generated 0 bytes in 813 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:24.972510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:24.974252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.099197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ee6174-8bb5-48b9-add1-d239a9d1dc5d service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:25.124488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:25.172826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:25.173660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.178411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.178411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] PATH_INFO: `/v3/users/a6009388b19d4068b8427d3e992acdc1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.178411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6009388b19d4068b8427d3e992acdc1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:25.178411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:25.212113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:25.214048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:25.218529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.237125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5b455-26ef-44c4-a1e5-fb47b7eee6fc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:25.251058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:25.251924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.252063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.252320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] PATH_INFO: `/v3/projects/7d124d74379246af91245cad86b056f5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.253698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d124d74379246af91245cad86b056f5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:25.255630 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:25.619289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d124d74379246af91245cad86b056f5', 'request_id': 'req-6d012d23-b892-40ab-a4de-382770ccbfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.623073 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6009388b19d4068b8427d3e992acdc1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.623866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6009388b19d4068b8427d3e992acdc1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.629517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d124d74379246af91245cad86b056f5', 'request_id': 'req-6d012d23-b892-40ab-a4de-382770ccbfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.641211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6009388b19d4068b8427d3e992acdc1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.642124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6009388b19d4068b8427d3e992acdc1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.657032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-16112966-ad25-478f-a809-56a65fac48c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6009388b19d4068b8427d3e992acdc1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.734069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d124d74379246af91245cad86b056f5', 'request_id': 'req-6d012d23-b892-40ab-a4de-382770ccbfde'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.746504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] The token cache is being invalidate because project 7d124d74379246af91245cad86b056f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:25.747545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.747545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6d012d23-b892-40ab-a4de-382770ccbfde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:25.749115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1085/2154] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:25 2022] DELETE /identity/v3/projects/7d124d74379246af91245cad86b056f5 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:25.755352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a05bff-e258-43fc-935b-374e2f44b657 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.755631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a05bff-e258-43fc-935b-374e2f44b657 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.755903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a05bff-e258-43fc-935b-374e2f44b657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.777578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c9a05bff-e258-43fc-935b-374e2f44b657 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:25.785492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c9a05bff-e258-43fc-935b-374e2f44b657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.832271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1070/2155] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:24 2022] DELETE /identity/v3/users/a6009388b19d4068b8427d3e992acdc1 => generated 0 bytes in 862 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:25.836881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.837216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.837497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.864265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:25.864852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.867600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1086/2156] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:25.877424 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:25.879089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:25.879089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:25.913612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:25.927747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:25.990044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1071/2157] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:26.004712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.007354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.075112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1087/2158] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:26.080587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.096367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e47ced-5b53-4201-95d4-6585bec189ea None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.099232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3031260a-edfc-412c-a952-2b6f88e427ea None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.104856 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.105518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.107325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.108170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.108296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.109099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.109389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.109843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.110810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:26.114157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.114157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.129525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1088/2159] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:26.133750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.136110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.138948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.155115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1072/2160] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:26.161503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e3c40-5c33-4722-ad81-a8b202b72555 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.179197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.179411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.181083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.181083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.181083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.182711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.322347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1089/2161] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/projects => generated 375 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:26.325654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.342735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e0952a-a387-4330-adbe-4f6d47bbb1ca service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.456292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.456992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.457499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.457939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.458658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.464296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.468809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:26.593733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1073/2162] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/users => generated 288 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:26.599287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.615053 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779e0981-e272-4025-8aab-9a93c27c430a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.620871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.621563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.621900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.622182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.622841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.624452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.637125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1090/2163] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:26.644147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.655403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e0a749-1d85-4551-8f84-2be4abd4e4a0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.660508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.661066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.661347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.661621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.662920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.665044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.676418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1074/2164] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:26.681310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.692329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0916c0df-cd90-4ba3-876b-495e383af89f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.697239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4638dd7-7d04-48b5-9aef-11a4c909414a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.698805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638dd7-7d04-48b5-9aef-11a4c909414a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.698805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638dd7-7d04-48b5-9aef-11a4c909414a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.698805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4638dd7-7d04-48b5-9aef-11a4c909414a admin admin] PATH_INFO: `/v3/projects/7aa16e85a90f44c4872c92d5db467f3e/users/55253e6867224ed78cca27eab3a40b68/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.741684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638dd7-7d04-48b5-9aef-11a4c909414a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7aa16e85a90f44c4872c92d5db467f3e, user_id=55253e6867224ed78cca27eab3a40b68, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.744862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4638dd7-7d04-48b5-9aef-11a4c909414a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:26.796384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:26.802958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.826436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1091/2165] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:26 2022] PUT /identity/v3/projects/7aa16e85a90f44c4872c92d5db467f3e/users/55253e6867224ed78cca27eab3a40b68/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:26.835657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7cdb1c-0a10-4e80-938a-38885f42adcc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:26.846525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.846819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.847412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.901833 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] MFA Rules not processed for user `55253e6867224ed78cca27eab3a40b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:26.903081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:26.965926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:26.967764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:26.967764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:26.967764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] PATH_INFO: `/v3/projects/d31853e070ac4ef0aeb6c17e9fbf9712` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:26.968530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d31853e070ac4ef0aeb6c17e9fbf9712)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:26.970760 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.021388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1092/2166] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:26 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:27.033903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.035406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.103099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:27.125249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd31853e070ac4ef0aeb6c17e9fbf9712', 'request_id': 'req-ace615a0-098a-4902-a738-c41948883c59'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.149419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fd3299-f43d-4994-8841-1085407172a3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.150319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd31853e070ac4ef0aeb6c17e9fbf9712', 'request_id': 'req-ace615a0-098a-4902-a738-c41948883c59'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.160670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd31853e070ac4ef0aeb6c17e9fbf9712', 'request_id': 'req-ace615a0-098a-4902-a738-c41948883c59'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.195634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:INSERT=2,keystone:DELETE=17 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:27.237743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.238444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.238799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.239201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.256640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] The token cache is being invalidate because project d31853e070ac4ef0aeb6c17e9fbf9712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:27.256640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.256640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.258021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1075/2167] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:26 2022] DELETE /identity/v3/projects/d31853e070ac4ef0aeb6c17e9fbf9712 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:27.263542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.265988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.380441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace615a0-098a-4902-a738-c41948883c59 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.388620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.389477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.389767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.390065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] PATH_INFO: `/v3/users/46d3266da86e41d9a0e9958ff36166fc` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.391366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=46d3266da86e41d9a0e9958ff36166fc)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.395444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.418191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.423486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.467341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1093/2168] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:27.630321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '46d3266da86e41d9a0e9958ff36166fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.631962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '46d3266da86e41d9a0e9958ff36166fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.661772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.664106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.668054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '46d3266da86e41d9a0e9958ff36166fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.674221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '46d3266da86e41d9a0e9958ff36166fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.680264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326a7e84-40da-4802-a62b-e0593bb2a7cf service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:27.691343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f2d1ed3c-c4e6-4e92-8665-a554ae50b132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '46d3266da86e41d9a0e9958ff36166fc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:27.696045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:27.696493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.696747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.697058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.700973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.821558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1076/2169] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:27 2022] DELETE /identity/v3/users/46d3266da86e41d9a0e9958ff36166fc => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:27.821558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9a5e-b127-47e2-8939-2abc720d1e87 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.821558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9a5e-b127-47e2-8939-2abc720d1e87 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.821558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bf9a5e-b127-47e2-8939-2abc720d1e87 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.821558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1077/2170] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:33:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:33:27.822031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:27.822031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:27.832948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1094/2171] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:27.852025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:27.852025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:27.852025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:27.880280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:27.882490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:27.980904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:27.982800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.005453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce16aef-8372-4cb0-af04-cd0146cd99c2 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.010665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1078/2172] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:33:27 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:28.022274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.023673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.037680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ec8913-d15d-4246-9404-aaa4f2b550a2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.050205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.050913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.051565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.051985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.063595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.072052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.078187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1079/2173] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:28.091126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.093442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.093442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.093442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] PATH_INFO: `/v3/users/1c7b02646d0d499b8928aed95f10e9ac` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.094306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c7b02646d0d499b8928aed95f10e9ac)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.096225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.344438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c7b02646d0d499b8928aed95f10e9ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.345374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c7b02646d0d499b8928aed95f10e9ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.346130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c7b02646d0d499b8928aed95f10e9ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.355210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c7b02646d0d499b8928aed95f10e9ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.368503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c7b02646d0d499b8928aed95f10e9ac'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.412267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.412267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.450319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea22f2d7-1d66-4e19-8120-73c2baf5c1b5 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.468405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.469957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.470262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.470547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] PATH_INFO: `/v3/projects/802fc643c1094af5a13f94a341f5def8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.474014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] RBAC: Authorizing `identity:delete_project(project_id=802fc643c1094af5a13f94a341f5def8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.476418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.477925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1095/2174] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:27 2022] DELETE /identity/v3/users/1c7b02646d0d499b8928aed95f10e9ac => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:28.508250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:28.672277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '802fc643c1094af5a13f94a341f5def8', 'request_id': 'req-0bdf4bdb-8388-4dc6-af37-914c614add39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.686079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '802fc643c1094af5a13f94a341f5def8', 'request_id': 'req-0bdf4bdb-8388-4dc6-af37-914c614add39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.695466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '802fc643c1094af5a13f94a341f5def8', 'request_id': 'req-0bdf4bdb-8388-4dc6-af37-914c614add39'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.723515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=3,keystone:DELETE=1,keystone:SELECT=96 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:28.848032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] The token cache is being invalidate because project 802fc643c1094af5a13f94a341f5def8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:28.848329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.849301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:28.851287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.851771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1080/2175] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:28 2022] DELETE /identity/v3/projects/802fc643c1094af5a13f94a341f5def8 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:28.853056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.856700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:28.858203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:28.936507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68587891-de1d-4232-bd2c-9f6e09a93846 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.943130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdf4bdb-8388-4dc6-af37-914c614add39 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:28.945332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.945332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.945586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.945758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] PATH_INFO: `/v3/projects/5882c9c6cf794b3ab17230304c169f07` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.947192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5882c9c6cf794b3ab17230304c169f07)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.949454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:28.951639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:28.952381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:28.952746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:28.953148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] PATH_INFO: `/v3/users/6282bbcd6a9d4ba5813cd44e6562f7c5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:28.954490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6282bbcd6a9d4ba5813cd44e6562f7c5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:28.956604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.100888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5882c9c6cf794b3ab17230304c169f07', 'request_id': 'req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.106628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5882c9c6cf794b3ab17230304c169f07', 'request_id': 'req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.201716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5882c9c6cf794b3ab17230304c169f07', 'request_id': 'req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.219577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6282bbcd6a9d4ba5813cd44e6562f7c5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.220339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6282bbcd6a9d4ba5813cd44e6562f7c5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.225191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] The token cache is being invalidate because project 5882c9c6cf794b3ab17230304c169f07 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:29.225712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.226740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2da7afa1-9d6f-4666-8558-4ce3ae6aa59c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.228616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1096/2176] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:28 2022] DELETE /identity/v3/projects/5882c9c6cf794b3ab17230304c169f07 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:29.233635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.235055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.235055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.241743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6282bbcd6a9d4ba5813cd44e6562f7c5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.242847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6282bbcd6a9d4ba5813cd44e6562f7c5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.252431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d9053beb-c72b-4a90-bb4f-5d8d362e91b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6282bbcd6a9d4ba5813cd44e6562f7c5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:29.277543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:29.278865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:29.352263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1081/2177] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:28 2022] DELETE /identity/v3/users/6282bbcd6a9d4ba5813cd44e6562f7c5 => generated 0 bytes in 496 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:29.368329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1097/2178] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:29.376607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.376986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.385795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.401337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:29.402630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:29.473133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1082/2179] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:29.477514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.488691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0e6af-d810-46ad-8cae-3ab9786c14d8 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.493841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.494453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.494731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.495774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.495774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:29.496047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:29.497732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.512231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1098/2180] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:29.516224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.526345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123fd20b-d36b-4952-8063-2df032c589aa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.531164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.531761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.532035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.532302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.532934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:29.534829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.628549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1083/2181] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:29 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:29.634339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.645792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36708c12-075a-4d84-aaea-3c1c172c5ed1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.719673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.720301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.720585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.721290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.721587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:29.723825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.842970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1099/2182] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:29 2022] POST /identity/v3/users => generated 295 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:29.848590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.865778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b960c0d-3190-4060-a682-63a05f8a1ff2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.871927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.872582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.872862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.873162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.873862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:29.875508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.890101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1084/2183] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:29.894728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.901666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.903448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:29.907814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46bca08-daf9-4931-8528-a378ea09acdf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.914402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.915024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.915411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.915692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.916345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:29.917981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.929994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1100/2184] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:29 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:29.935808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:29.946821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e941c472-3249-4b20-a98d-7b85ea114404 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.951667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d4acf-f042-4646-8688-92bbc0446527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.952335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d4acf-f042-4646-8688-92bbc0446527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.952616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d4acf-f042-4646-8688-92bbc0446527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.952982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d4acf-f042-4646-8688-92bbc0446527 admin admin] PATH_INFO: `/v3/projects/b02af2d33ca74bd08365dfc34a6fa77a/users/3eda1fca08b24727929b63318fd6f13e/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.967996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ab67a4-e6c3-48ae-82a2-9273cf732c95 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:29.971399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d4acf-f042-4646-8688-92bbc0446527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b02af2d33ca74bd08365dfc34a6fa77a, user_id=3eda1fca08b24727929b63318fd6f13e, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:29.971733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:29.972596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:29.972875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:29.973182 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:29.974415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d4acf-f042-4646-8688-92bbc0446527 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:29.975135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.051712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:30.053275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:30.063022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1085/2185] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:30.063837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:30.065235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.081134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49553bc9-33ca-48ad-996e-17b2f02990e9 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:30.086511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1101/2186] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:29 2022] PUT /identity/v3/projects/b02af2d33ca74bd08365dfc34a6fa77a/users/3eda1fca08b24727929b63318fd6f13e/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:30.089491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.089873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.090268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.112180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:30.120515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.157262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:30.158199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.158430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.158683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] PATH_INFO: `/v3/projects/e1a97977852d404eada3674347636061` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.160022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1a97977852d404eada3674347636061)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:30.162323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:30.197811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1102/2187] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:30.204885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:30.206620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.287496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0188de3-85ed-421a-83a3-6665d0949990 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:30.293925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af19b20-a38f-47e0-901e-5ac1c66a0223 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:30.294568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af19b20-a38f-47e0-901e-5ac1c66a0223 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.294841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af19b20-a38f-47e0-901e-5ac1c66a0223 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.295122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af19b20-a38f-47e0-901e-5ac1c66a0223 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.311107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af19b20-a38f-47e0-901e-5ac1c66a0223 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:30.312592 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af19b20-a38f-47e0-901e-5ac1c66a0223 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:30.325119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1103/2188] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:30.372687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:30.393867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1a97977852d404eada3674347636061', 'request_id': 'req-b6db937f-e9c4-4b08-b61f-12306517029e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:30.413437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1a97977852d404eada3674347636061', 'request_id': 'req-b6db937f-e9c4-4b08-b61f-12306517029e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:30.420179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1a97977852d404eada3674347636061', 'request_id': 'req-b6db937f-e9c4-4b08-b61f-12306517029e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:30.425721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691becc9-e8b3-445b-b767-e8a1447f6359 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.426012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691becc9-e8b3-445b-b767-e8a1447f6359 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.426274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691becc9-e8b3-445b-b767-e8a1447f6359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.448730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-691becc9-e8b3-445b-b767-e8a1447f6359 None None] MFA Rules not processed for user `55253e6867224ed78cca27eab3a40b68`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:30.452740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-691becc9-e8b3-445b-b767-e8a1447f6359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.511665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1104/2189] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:30.513916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] The token cache is being invalidate because project e1a97977852d404eada3674347636061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:30.514374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:30.515041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-b6db937f-e9c4-4b08-b61f-12306517029e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:30.516699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1086/2190] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:30 2022] DELETE /identity/v3/projects/e1a97977852d404eada3674347636061 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:30.531356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.532013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.532346 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.556365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:30.557775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.605869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:30.628104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1105/2191] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:30.638986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.638986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.638986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.660495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:30.661666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:30.674394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1087/2192] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:30.679290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:30.690319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd52a90-ce88-498e-84a7-05fdb1c7fb94 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:30.696090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:30.696746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.698062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.698353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.699101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:30.699415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:30.701324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:30.715205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1106/2193] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:30.719956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:30.739816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadbcb36-8ca7-46ba-a896-87fc2bbd223b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:30.748199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:30.748945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:30.749441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:30.749774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:30.751033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:30.752415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:30.914599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1088/2194] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:30 2022] POST /identity/v3/projects => generated 383 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:30.919577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:30.933004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd74ff4-5455-4a09-8263-58a47e48ec12 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.001550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.002082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.002364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.002635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.003275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.005294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.035714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:31.105870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1107/2195] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:30 2022] POST /identity/v3/users => generated 292 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:31.110067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.121234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52de5f3f-4907-48b1-bef4-5879dca396fa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.126919 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.127609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.127890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.130111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.130111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.130485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.142451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1089/2196] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:31.146747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.159205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e32ba9-871d-4647-a413-921818a76812 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.164818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f4177d-3c6d-47db-ba27-3d5d93789f9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.165549 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4177d-3c6d-47db-ba27-3d5d93789f9e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.165828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4177d-3c6d-47db-ba27-3d5d93789f9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.166103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f4177d-3c6d-47db-ba27-3d5d93789f9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.166755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4177d-3c6d-47db-ba27-3d5d93789f9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.168358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f4177d-3c6d-47db-ba27-3d5d93789f9e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.181262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1108/2197] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:31.185673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.196915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b0c570-77fa-4020-8af1-b40ba8df44b1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.201949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.202957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.203249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.203527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] PATH_INFO: `/v3/projects/0072040cdf534fa6a1adfd3b00a9f6a5/users/5a5d0b59eac74fe78cd444003f588b25/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.220397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0072040cdf534fa6a1adfd3b00a9f6a5, user_id=5a5d0b59eac74fe78cd444003f588b25, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.223743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.288795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1090/2198] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:31 2022] PUT /identity/v3/projects/0072040cdf534fa6a1adfd3b00a9f6a5/users/5a5d0b59eac74fe78cd444003f588b25/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:31.300228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.300508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.300785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.334830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] MFA Rules not processed for user `5a5d0b59eac74fe78cd444003f588b25`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:31.334830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:31.408791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1109/2199] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:31 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:31.408983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.413800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:31.517607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb0b19-3427-4c67-b9c8-56a92c6f3b6d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.525470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.526106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.526855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.526855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.527387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.529310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.608167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:31.617617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1091/2200] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:31 2022] POST /identity/v3/projects => generated 385 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:31.633147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.643413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8794a964-25e2-4e8a-8789-41acc9df41e6 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.688974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:DELETE=3,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:31.724165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.734810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.734810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.734810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.734810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.734810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.882901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1110/2201] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:31 2022] POST /identity/v3/users => generated 299 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:31.886815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.898454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0b395f-19fa-4b6b-b2d7-923dda8352eb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.903528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.904169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.904454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.904726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.905545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.907271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:31.920459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1092/2202] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:31.924946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:31.935661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9f909-fc1f-477e-9489-a33b2930e1b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:31.941269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:31.941975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:31.942255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:31.942525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] PATH_INFO: `/v3/projects/4fd162af736b4a5fb4367f4241d040ad/users/86103d2ae8a542ee9207e46d292c3008/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:31.959333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd162af736b4a5fb4367f4241d040ad, user_id=86103d2ae8a542ee9207e46d292c3008, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:31.962231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:32.106601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1111/2203] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:31 2022] PUT /identity/v3/projects/4fd162af736b4a5fb4367f4241d040ad/users/86103d2ae8a542ee9207e46d292c3008/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:32.111091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:32.122210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90e40a6-3bdb-4e71-9d85-4e5a5eefd97c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:32.122954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:32.184628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7121cb74-c428-499b-ad70-35f07f32a892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:32.185278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cb74-c428-499b-ad70-35f07f32a892 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:32.185563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cb74-c428-499b-ad70-35f07f32a892 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:32.185867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7121cb74-c428-499b-ad70-35f07f32a892 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:32.186519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cb74-c428-499b-ad70-35f07f32a892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:32.188108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7121cb74-c428-499b-ad70-35f07f32a892 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:32.199214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1093/2204] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:32.203739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:32.214647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d67908a-acdd-41d5-a431-5849029d4f6f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:32.219456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:32.220122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:32.220403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:32.220673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] PATH_INFO: `/v3/projects/4fd162af736b4a5fb4367f4241d040ad/users/86103d2ae8a542ee9207e46d292c3008/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:32.222942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4fd162af736b4a5fb4367f4241d040ad, user_id=86103d2ae8a542ee9207e46d292c3008, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:32.225625 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:32.236190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1112/2205] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:32 2022] PUT /identity/v3/projects/4fd162af736b4a5fb4367f4241d040ad/users/86103d2ae8a542ee9207e46d292c3008/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:32.248414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d9f3-af24-4540-a195-0603a021c731 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:32.250264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d9f3-af24-4540-a195-0603a021c731 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:32.250264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f3d9f3-af24-4540-a195-0603a021c731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:32.293379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-02f3d9f3-af24-4540-a195-0603a021c731 None None] MFA Rules not processed for user `86103d2ae8a542ee9207e46d292c3008`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:32.294686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-02f3d9f3-af24-4540-a195-0603a021c731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:32.375505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1094/2206] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:32 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:32.392726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:32.394505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:32.408235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:32.469105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cad21d-d762-4c96-a068-155760c092bd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:32.473487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:32.474105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:32.474494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:32.474960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:32.543268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:32.544754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:32.561557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1113/2207] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:32 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:32.603690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:33.340255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:37.448840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:37.449363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:37.449472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:37.477761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:37.503290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] MFA Rules not processed for user `3eda1fca08b24727929b63318fd6f13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:37.509715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:37.606509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1095/2208] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:37 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:37.616687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:37.619118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:37.824741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c9ce6e-2ecb-4e11-98a9-c36b9e1cba07 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:37.831824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa19a10-caba-4a9f-ad00-55584f24e59c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:37.833286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19a10-caba-4a9f-ad00-55584f24e59c service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:37.834431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19a10-caba-4a9f-ad00-55584f24e59c service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:37.834880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa19a10-caba-4a9f-ad00-55584f24e59c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:37.845639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa19a10-caba-4a9f-ad00-55584f24e59c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:37.848373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa19a10-caba-4a9f-ad00-55584f24e59c service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:37.895727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1114/2209] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:37 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:42.284954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:42.288582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:42.391213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953f7d3-5dad-4ba8-a3bb-adbad3357eda None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:42.405402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:42.405402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:42.405402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:42.405402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] PATH_INFO: `/v3/users/76307b04d59a4db4bcb65fb5288ed07b` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:42.406403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76307b04d59a4db4bcb65fb5288ed07b)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:42.411243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10353ba6-b5d2-434a-a60f-6451cbe69367 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:42.412453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10353ba6-b5d2-434a-a60f-6451cbe69367 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:42.412641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10353ba6-b5d2-434a-a60f-6451cbe69367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:42.414564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:42.473454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-10353ba6-b5d2-434a-a60f-6451cbe69367 None None] MFA Rules not processed for user `3eda1fca08b24727929b63318fd6f13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:42.476163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10353ba6-b5d2-434a-a60f-6451cbe69367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:42.485080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1115/2210] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:42.971125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76307b04d59a4db4bcb65fb5288ed07b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:42.971125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76307b04d59a4db4bcb65fb5288ed07b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:42.993662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76307b04d59a4db4bcb65fb5288ed07b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:42.995613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76307b04d59a4db4bcb65fb5288ed07b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.018508 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76307b04d59a4db4bcb65fb5288ed07b'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.165704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1096/2211] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:42 2022] DELETE /identity/v3/users/76307b04d59a4db4bcb65fb5288ed07b => generated 0 bytes in 878 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:43.167331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.167847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.168279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.207098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:43.211967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.332733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1116/2212] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:43.343500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:43.345103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.373548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:43.437966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:43.440363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.460899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e00b83a-8684-4e64-9126-5b2b29565d06 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:43.470036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:43.470901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.471840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.472271 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.494735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:43.500708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:43.510320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e847f0e-9e6e-487c-9626-2780c8bf3fc2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:43.515297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1097/2213] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:43.531051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:43.532729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.533647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.536372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] PATH_INFO: `/v3/users/bf7c7782806442f2971a6b583a66b1c2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.540010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7c7782806442f2971a6b583a66b1c2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:43.543757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:43.753359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:43.755817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:43.871807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7c7782806442f2971a6b583a66b1c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.874242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7c7782806442f2971a6b583a66b1c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.875115 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7c7782806442f2971a6b583a66b1c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.915336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7c7782806442f2971a6b583a66b1c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.956487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c175bc-d86d-402f-a37a-323b47e5e547 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:43.959737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:43.962618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-edfe8aa2-3614-4a48-8c83-a60e25568e94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7c7782806442f2971a6b583a66b1c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:43.963031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:43.964275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:43.964552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:43.964828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:43.969987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.058670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1117/2214] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:43 2022] DELETE /identity/v3/users/bf7c7782806442f2971a6b583a66b1c2 => generated 0 bytes in 622 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:44.064061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.064731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.072008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.135770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:44.137114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.138846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.140470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.171039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1098/2215] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:43 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:44.229371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1118/2216] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:44.240123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.241616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.257536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1397ef-46b9-4270-82ae-df5fbbd518c7 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.326999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.327702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.328013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.328264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.339121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.340585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.351538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1099/2217] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:44.377635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.379844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:44.394277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73c2cc6-7392-48b3-8167-7ccbdaa9208e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.403990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.403990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.403990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.403990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] PATH_INFO: `/v3/projects/53e9df6b688945d09b4d23f3f812d33b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.404348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] RBAC: Authorizing `identity:delete_project(project_id=53e9df6b688945d09b4d23f3f812d33b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.405610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:44.611446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53e9df6b688945d09b4d23f3f812d33b', 'request_id': 'req-08e5ef2f-73c1-4561-9185-f52daae0e15d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.626318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53e9df6b688945d09b4d23f3f812d33b', 'request_id': 'req-08e5ef2f-73c1-4561-9185-f52daae0e15d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.749811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53e9df6b688945d09b4d23f3f812d33b', 'request_id': 'req-08e5ef2f-73c1-4561-9185-f52daae0e15d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.769541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] The token cache is being invalidate because project 53e9df6b688945d09b4d23f3f812d33b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:44.769835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.770682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:44.773728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1119/2218] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:44 2022] DELETE /identity/v3/projects/53e9df6b688945d09b4d23f3f812d33b => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:44.778449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:44.794838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3824bd7c-f941-499e-a3d9-c1d0eea3b59b service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:44.877840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:44.878901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:44.879206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:44.879509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] PATH_INFO: `/v3/users/5a03bfcbbca74a738bed8d0b1cca0854` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:44.881706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a03bfcbbca74a738bed8d0b1cca0854)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:44.884358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.019370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.023687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:45.072015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e5ef2f-73c1-4561-9185-f52daae0e15d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:45.085569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:45.089828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.089828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.089828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] PATH_INFO: `/v3/projects/3740344ed3d44994a8b3ef9cdbb69bf5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.089828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3740344ed3d44994a8b3ef9cdbb69bf5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:45.090216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:45.182473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a03bfcbbca74a738bed8d0b1cca0854'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.183216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a03bfcbbca74a738bed8d0b1cca0854'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.204937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a03bfcbbca74a738bed8d0b1cca0854'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.206075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a03bfcbbca74a738bed8d0b1cca0854'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.218836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a03bfcbbca74a738bed8d0b1cca0854'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.271681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3740344ed3d44994a8b3ef9cdbb69bf5', 'request_id': 'req-87e3f119-4dff-49b3-8369-dea8bb34ea31'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.279588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3740344ed3d44994a8b3ef9cdbb69bf5', 'request_id': 'req-87e3f119-4dff-49b3-8369-dea8bb34ea31'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.315984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1100/2219] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:44 2022] DELETE /identity/v3/users/5a03bfcbbca74a738bed8d0b1cca0854 => generated 0 bytes in 539 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:45.448205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3740344ed3d44994a8b3ef9cdbb69bf5', 'request_id': 'req-87e3f119-4dff-49b3-8369-dea8bb34ea31'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.466458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] The token cache is being invalidate because project 3740344ed3d44994a8b3ef9cdbb69bf5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:45.466673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.467331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:45.472723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1120/2220] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:45 2022] DELETE /identity/v3/projects/3740344ed3d44994a8b3ef9cdbb69bf5 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:45.474529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:45.476408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:45.571827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39469b01-bc29-4693-a8c0-656cea108669 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:45.576752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:45.577429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:45.577886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:45.578140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] PATH_INFO: `/v3/users/c906e0990ba14e688ab3c0b39449b42f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:45.590574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=c906e0990ba14e688ab3c0b39449b42f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:45.590574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.025772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.027307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:46.034860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c906e0990ba14e688ab3c0b39449b42f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.035917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c906e0990ba14e688ab3c0b39449b42f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.093177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c906e0990ba14e688ab3c0b39449b42f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.094672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c906e0990ba14e688ab3c0b39449b42f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.131169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c906e0990ba14e688ab3c0b39449b42f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.166589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e3f119-4dff-49b3-8369-dea8bb34ea31 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.175698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.177504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.177979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.178688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] PATH_INFO: `/v3/projects/344e1abfbee4401c95aad4330eedd5f0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.184516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=344e1abfbee4401c95aad4330eedd5f0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.188496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.304408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1101/2221] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:45 2022] DELETE /identity/v3/users/c906e0990ba14e688ab3c0b39449b42f => generated 0 bytes in 822 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:46.391963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '344e1abfbee4401c95aad4330eedd5f0', 'request_id': 'req-3930ac2e-cc03-478d-b0a7-b022e68671c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.399063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '344e1abfbee4401c95aad4330eedd5f0', 'request_id': 'req-3930ac2e-cc03-478d-b0a7-b022e68671c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.430728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=82,keystone:INSERT=3,keystone:DELETE=15 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:46.601759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '344e1abfbee4401c95aad4330eedd5f0', 'request_id': 'req-3930ac2e-cc03-478d-b0a7-b022e68671c2'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.614540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] The token cache is being invalidate because project 344e1abfbee4401c95aad4330eedd5f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:46.614977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.615881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:46.617645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1121/2222] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:46 2022] DELETE /identity/v3/projects/344e1abfbee4401c95aad4330eedd5f0 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:46.622181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.623822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:46.683730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:46.730589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd4a558-2f11-4367-af43-a8039fd4d1fa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.804007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.804650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.804929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.805263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] PATH_INFO: `/v3/users/3ecdf9deaa1c448bb16f0db36347cdc4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.806408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ecdf9deaa1c448bb16f0db36347cdc4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.813205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:46.948604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:46.951233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:46.976665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3930ac2e-cc03-478d-b0a7-b022e68671c2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:46.982248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:46.982916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:46.983367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:46.983641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] PATH_INFO: `/v3/projects/5d030540d70846f59b10ce57f31ad0dc` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:46.985830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d030540d70846f59b10ce57f31ad0dc)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:46.988273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.050064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ecdf9deaa1c448bb16f0db36347cdc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.050758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ecdf9deaa1c448bb16f0db36347cdc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.064392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ecdf9deaa1c448bb16f0db36347cdc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.065247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ecdf9deaa1c448bb16f0db36347cdc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.073794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-83fb13b1-053b-4737-8675-10dfdd7ffd69 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ecdf9deaa1c448bb16f0db36347cdc4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.169472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d030540d70846f59b10ce57f31ad0dc', 'request_id': 'req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.182979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1102/2223] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:46 2022] DELETE /identity/v3/users/3ecdf9deaa1c448bb16f0db36347cdc4 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:47.202126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d030540d70846f59b10ce57f31ad0dc', 'request_id': 'req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.287184 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d030540d70846f59b10ce57f31ad0dc', 'request_id': 'req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.306659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] The token cache is being invalidate because project 5d030540d70846f59b10ce57f31ad0dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:47.330646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.350949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-98c2f3c5-f5da-490f-9fc1-9e8bc781f9dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:47.354434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1122/2224] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:46 2022] DELETE /identity/v3/projects/5d030540d70846f59b10ce57f31ad0dc => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:47.361451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.362020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.362452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.397270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:47.399221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:47.486441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1103/2225] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:47.498916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.501835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.501835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.530389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:47.531782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:47.543272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1123/2226] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:47.547503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.570358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbce9f3d-9aa7-4720-acf2-820f2f05f054 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.579130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.581223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.582218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.582890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.585862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:47.586178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.589291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.605244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1104/2227] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:47.609218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.619519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047e776b-a99a-4969-8d36-62ee11170cb2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.625340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.626237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.626664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.627122 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.627850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.630284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.780561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1124/2228] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/projects => generated 381 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:47.785395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.797201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13eeeda1-1f6a-4a85-8575-6a33be8a2d19 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.820156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:47.834064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:47.931978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.932912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.933366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.934064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.934716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.937333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:47.944145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:47.945858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e9dca-7d44-434d-9de9-6028c5ce2d2d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:47.951443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:47.952295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:47.952659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:47.953324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] PATH_INFO: `/v3/projects/b0670ba65aa94f85bd9c7bfd6220f715` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:47.954695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0670ba65aa94f85bd9c7bfd6220f715)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:47.956661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.064635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1105/2229] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:47 2022] POST /identity/v3/users => generated 291 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:48.069231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.079395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d775801-ddc8-4e9d-8ad8-221e19caed37 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.084785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.085460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.085799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.085982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.086630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.088241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.100353 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1106/2230] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:48.105640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.105875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0670ba65aa94f85bd9c7bfd6220f715', 'request_id': 'req-e26c3840-0e50-4633-902d-b63fb49ca91f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:48.115217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0670ba65aa94f85bd9c7bfd6220f715', 'request_id': 'req-e26c3840-0e50-4633-902d-b63fb49ca91f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:48.117913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9a0e8f-4ced-4048-b948-7cce358708ea admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.179219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.179850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.180133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.180403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.181387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.188209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.195392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1107/2231] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:48.200023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.212731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ae3616-a7dc-4134-aaab-7343dca32cb5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.229203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0670ba65aa94f85bd9c7bfd6220f715', 'request_id': 'req-e26c3840-0e50-4633-902d-b63fb49ca91f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:48.229759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23e629b-2ebe-4228-a590-389241f8b992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.229759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e629b-2ebe-4228-a590-389241f8b992 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.229759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e629b-2ebe-4228-a590-389241f8b992 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.229759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23e629b-2ebe-4228-a590-389241f8b992 admin admin] PATH_INFO: `/v3/projects/f0c9ff1e74f74e1b85d0c7e03c238283/users/bf7b71625aca410faa17bd11a7879750/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.245395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] The token cache is being invalidate because project b0670ba65aa94f85bd9c7bfd6220f715 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:48.245395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:48.245395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e26c3840-0e50-4633-902d-b63fb49ca91f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:48.245395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1125/2232] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:47 2022] DELETE /identity/v3/projects/b0670ba65aa94f85bd9c7bfd6220f715 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:48.251456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4bc06-ca13-4938-ba56-9464dc64f2fe None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.251880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4bc06-ca13-4938-ba56-9464dc64f2fe None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.252650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a4bc06-ca13-4938-ba56-9464dc64f2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.253406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e629b-2ebe-4228-a590-389241f8b992 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f0c9ff1e74f74e1b85d0c7e03c238283, user_id=bf7b71625aca410faa17bd11a7879750, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.257429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d23e629b-2ebe-4228-a590-389241f8b992 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.277673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f2a4bc06-ca13-4938-ba56-9464dc64f2fe None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:48.279087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f2a4bc06-ca13-4938-ba56-9464dc64f2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:48.287832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1126/2233] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:48.296142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.296492 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.296692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.319006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:48.320251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:48.328803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1127/2234] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:48.333203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.342764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa0401-4a6a-47a0-a9ac-88dba383863f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.347946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.348585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.348872 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.349180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.350057 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:48.350365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.351979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.364807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1128/2235] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:48.369407 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.382107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da03173-c67f-4f45-bdc7-8eabc3d711b0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.398425 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.400819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.401247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.401645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.403216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.410911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1108/2236] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:48 2022] PUT /identity/v3/projects/f0c9ff1e74f74e1b85d0c7e03c238283/users/bf7b71625aca410faa17bd11a7879750/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:48.411088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.411189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.411189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.411189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.463068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] MFA Rules not processed for user `bf7b71625aca410faa17bd11a7879750`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:48.465887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:48.556311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1129/2237] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/projects => generated 371 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:48.564046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.585185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1109/2238] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:48.586293 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.587835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:48.688969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1195dd98-c20e-47a6-a134-f0ee41650e29 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.697262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.698007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.698305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.698685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.699266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.701302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.730369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f3e2c-d849-49ac-9c36-1ea8ed6dc9a2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.738611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.739319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.739661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.739987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.740656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.742800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.884949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1130/2239] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/users => generated 286 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:48.886842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.899132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a8a8b9-358e-4ef2-bd98-0436c6173e03 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.904912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.905572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.905856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.906132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.906984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.909691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.923009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1131/2240] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:48.927650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.938136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90351b95-a1ee-4fa4-87a9-c5bbf18756c7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.945470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:48.946103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:48.946386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:48.946659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:48.947311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:48.949006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:48.961772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1132/2241] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:48.966756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:48.983906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d650f7-bab8-4403-a8c2-072dd5424d5f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:48.987472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1110/2242] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/projects => generated 381 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:48.993784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:49.000233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea0869-3a08-4f27-864d-73a36ad61a43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.001207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea0869-3a08-4f27-864d-73a36ad61a43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.001745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea0869-3a08-4f27-864d-73a36ad61a43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.002033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea0869-3a08-4f27-864d-73a36ad61a43 admin admin] PATH_INFO: `/v3/projects/82385f33e9134e00a9984f7e83390ba8/users/6dfc723a05e74342ab277d77d02317c1/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.031367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea0869-3a08-4f27-864d-73a36ad61a43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82385f33e9134e00a9984f7e83390ba8, user_id=6dfc723a05e74342ab277d77d02317c1, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.039265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea0869-3a08-4f27-864d-73a36ad61a43 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.120509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ac61be-e67a-4c92-8f43-b1cf2be8c442 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:49.125769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.126726 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.127094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.127431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.128142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.130621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.175426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1133/2243] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:48 2022] PUT /identity/v3/projects/82385f33e9134e00a9984f7e83390ba8/users/6dfc723a05e74342ab277d77d02317c1/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:49.187520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.188059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.188428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.234930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] MFA Rules not processed for user `6dfc723a05e74342ab277d77d02317c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:49.236351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:49.281084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1111/2244] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:48 2022] POST /identity/v3/users => generated 297 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:49.284938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:49.337383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c336ea71-f5eb-43f6-a2dc-4dc4c5397c84 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:49.472447 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1134/2245] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:49.480810 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:49.488720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:49.488956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.488956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.488956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.488956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.491447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.505557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.536241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1112/2246] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:49.553558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:49.579072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd4a57f-1024-49d7-a204-2359e696ce73 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:49.601881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.602969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.603481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.604040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] PATH_INFO: `/v3/projects/7f90e344247049fb84cb40ccf41ef7d2/users/22d11d4bfe96474d94e90bc4cb5db80f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.611137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10b3606-f6c6-4f0d-aeec-0a14ecd57c3d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:49.617378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.618438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.618917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.619386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.633148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.635231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.661171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1135/2247] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:49.667745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f90e344247049fb84cb40ccf41ef7d2, user_id=22d11d4bfe96474d94e90bc4cb5db80f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.679657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.809378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1113/2248] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:49 2022] PUT /identity/v3/projects/7f90e344247049fb84cb40ccf41ef7d2/users/22d11d4bfe96474d94e90bc4cb5db80f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:49.810338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:49.823772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c502040c-e2f9-463d-8a61-fd40934ed36f service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:49.918880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c855c11-f18f-4771-8bc2-9d9d87dd6d2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.919516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855c11-f18f-4771-8bc2-9d9d87dd6d2c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.919798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855c11-f18f-4771-8bc2-9d9d87dd6d2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.920093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c855c11-f18f-4771-8bc2-9d9d87dd6d2c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.920732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c855c11-f18f-4771-8bc2-9d9d87dd6d2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.922432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c855c11-f18f-4771-8bc2-9d9d87dd6d2c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.941061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1136/2249] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:33:49.945668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:49.959064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e469f3-821c-46af-84bf-424ed6b6fe6f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:49.964292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:49.965036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:49.965381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:49.965757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] PATH_INFO: `/v3/projects/7f90e344247049fb84cb40ccf41ef7d2/users/22d11d4bfe96474d94e90bc4cb5db80f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:49.968132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f90e344247049fb84cb40ccf41ef7d2, user_id=22d11d4bfe96474d94e90bc4cb5db80f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:49.971779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:49.987893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1114/2250] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:49 2022] PUT /identity/v3/projects/7f90e344247049fb84cb40ccf41ef7d2/users/22d11d4bfe96474d94e90bc4cb5db80f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:50.011228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb2697-1e5c-40da-a779-0096ed12b1e6 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:50.011848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb2697-1e5c-40da-a779-0096ed12b1e6 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:50.012345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19eb2697-1e5c-40da-a779-0096ed12b1e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:50.064989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-19eb2697-1e5c-40da-a779-0096ed12b1e6 None None] MFA Rules not processed for user `22d11d4bfe96474d94e90bc4cb5db80f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:50.066781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-19eb2697-1e5c-40da-a779-0096ed12b1e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:50.174564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1137/2251] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:50 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:50.188950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:50.190813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:50.272417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff239d7-418f-49a5-b299-35392c051c97 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:50.276391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:50.285425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:50.285425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:50.285425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:50.291247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:50.292680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:50.303256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1115/2252] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:50 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:50.320129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:50.320586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:50.320677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:50.341884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] MFA Rules not processed for user `ebbf49534a8f41109f51637307270016`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:50.343241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:50.477835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1138/2253] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:50 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:50.510908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:51.705900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:51.790125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:51.803388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f13294-f296-4338-b2e9-b01b9e582843 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:51.808672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22d11d4bfe96474d94e90bc4cb5db80f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f90e344247049fb84cb40ccf41ef7d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:51.809392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:51.809740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:51.810082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:51.810804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:51.811177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:51.812897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:51.826910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1116/2254] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:51.831257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:51.844236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1990ee3b-79c8-4b48-8037-c6819c0b31bc None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:51.852690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22d11d4bfe96474d94e90bc4cb5db80f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f90e344247049fb84cb40ccf41ef7d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:51.853360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:51.853652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:51.853933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:51.854576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:51.856501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:52.015274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1139/2255] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:51 2022] POST /identity/v3/projects => generated 355 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:33:52.016565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:52.028344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9018028-d11e-4a85-a40d-a28f3429fcc8 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:52.139272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22d11d4bfe96474d94e90bc4cb5db80f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f90e344247049fb84cb40ccf41ef7d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:52.140358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:52.140861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:52.141474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:52.142672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:52.143223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:52.146502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:52.174786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1117/2256] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 22:33:52.227490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:52.240005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f2e7e6-3097-41ef-86c2-0dcd78595ed0 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:52.245505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22d11d4bfe96474d94e90bc4cb5db80f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7f90e344247049fb84cb40ccf41ef7d2', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:52.246082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:52.246361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:52.246652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] PATH_INFO: `/v3/projects/1148c68045934634933d6c144b207818` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:52.247928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorizing `identity:delete_project(project_id=1148c68045934634933d6c144b207818)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:52.250008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:52.401000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1148c68045934634933d6c144b207818', 'request_id': 'req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:52.408190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1148c68045934634933d6c144b207818', 'request_id': 'req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:52.539124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1148c68045934634933d6c144b207818', 'request_id': 'req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:52.566277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] The token cache is being invalidate because project 1148c68045934634933d6c144b207818 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:52.566571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:52.567341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:52.572885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1140/2257] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:52 2022] DELETE /identity/v3/projects/1148c68045934634933d6c144b207818 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:53.304857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:53.518495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:53.521077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:53.539082 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:53.540639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:53.635749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29d5d1d-9796-4219-8988-5a8b47a8e983 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:53.636041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9b2d1f-57b6-4f26-9b47-c12471714ff6 tempest-VolumeQuotaClassesTest-1431312554 tempest-VolumeQuotaClassesTest-1431312554-project-admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:53.641576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:53.642571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:53.642853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:53.643649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] PATH_INFO: `/v3/users/bf7b71625aca410faa17bd11a7879750` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:53.644890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf7b71625aca410faa17bd11a7879750)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:53.647158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:53.647765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:53.647887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:53.648083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:53.648363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] PATH_INFO: `/v3/users/5a5d0b59eac74fe78cd444003f588b25` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:53.649553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a5d0b59eac74fe78cd444003f588b25)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:53.652221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:54.006366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a5d0b59eac74fe78cd444003f588b25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.007147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a5d0b59eac74fe78cd444003f588b25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.009313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf7b71625aca410faa17bd11a7879750'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.009970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf7b71625aca410faa17bd11a7879750'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.010574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf7b71625aca410faa17bd11a7879750'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.018195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf7b71625aca410faa17bd11a7879750'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.025206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a5d0b59eac74fe78cd444003f588b25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.025954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a5d0b59eac74fe78cd444003f588b25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.032079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b1515dbe-2a10-4ca7-82c2-2ed50b2cdec4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf7b71625aca410faa17bd11a7879750'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.035462 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-ffa06de4-9601-45a6-8717-877bd38ba04b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a5d0b59eac74fe78cd444003f588b25'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:54.256543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1141/2258] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:53 2022] DELETE /identity/v3/users/bf7b71625aca410faa17bd11a7879750 => generated 0 bytes in 714 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:54.258141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.263965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.263965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:54.278974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:54.280209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:54.326523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1118/2259] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:53 2022] DELETE /identity/v3/users/5a5d0b59eac74fe78cd444003f588b25 => generated 0 bytes in 809 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:54.331288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.331580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.332007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:54.347912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1142/2260] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:54.354800 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:54.356213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:54.357894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:54.359423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:54.420601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1119/2261] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:54.428042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711a3a3c-0da7-4503-824b-e9877108b70c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:54.431188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:54.490294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90dd555-a03d-46d9-bbe5-3750146e69b1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:54.496164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:54.496705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.496978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.497282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:54.502520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:54.503508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:54.504224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:54.504398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:54.508678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:54.510156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:54.516524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:54.517550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:54.518019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:54.524255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1120/2262] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:54.529812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1143/2263] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:55.343408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:55.344394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:55.345341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.356537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.360464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fef7d-a6ff-4df5-9dd4-3452c2384625 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:55.366319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:55.366741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:55.367022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:55.367904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] PATH_INFO: `/v3/projects/0072040cdf534fa6a1adfd3b00a9f6a5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.368623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0072040cdf534fa6a1adfd3b00a9f6a5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:55.370585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:55.391108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987be15f-f9ac-4b04-a0cb-26916f088209 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:55.404138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:55.405338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:55.406261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:55.407816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] PATH_INFO: `/v3/projects/f0c9ff1e74f74e1b85d0c7e03c238283` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.409999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] RBAC: Authorizing `identity:delete_project(project_id=f0c9ff1e74f74e1b85d0c7e03c238283)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:55.412440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:55.596358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f0c9ff1e74f74e1b85d0c7e03c238283', 'request_id': 'req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.596542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0072040cdf534fa6a1adfd3b00a9f6a5', 'request_id': 'req-efeadfdf-f0a9-4553-bbac-b1d12be690a1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.602172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f0c9ff1e74f74e1b85d0c7e03c238283', 'request_id': 'req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.602795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0072040cdf534fa6a1adfd3b00a9f6a5', 'request_id': 'req-efeadfdf-f0a9-4553-bbac-b1d12be690a1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.609811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0072040cdf534fa6a1adfd3b00a9f6a5', 'request_id': 'req-efeadfdf-f0a9-4553-bbac-b1d12be690a1'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.658650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f0c9ff1e74f74e1b85d0c7e03c238283', 'request_id': 'req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.672431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] The token cache is being invalidate because project f0c9ff1e74f74e1b85d0c7e03c238283 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:55.672723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.673438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.675170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1144/2264] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:55 2022] DELETE /identity/v3/projects/f0c9ff1e74f74e1b85d0c7e03c238283 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:55.680219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:55.681599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.750264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adef6d4b-5ba3-4ebe-b3ad-63a4796ec05b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:55.756419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:55.756642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] The token cache is being invalidate because project 0072040cdf534fa6a1adfd3b00a9f6a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:55.756774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:55.756972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.757511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:55.757511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] PATH_INFO: `/v3/users/22d11d4bfe96474d94e90bc4cb5db80f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.762302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:55.762453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22d11d4bfe96474d94e90bc4cb5db80f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:55.762552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1121/2265] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:55 2022] DELETE /identity/v3/projects/0072040cdf534fa6a1adfd3b00a9f6a5 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:55.762644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:55.763954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:55.765424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:55.842868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeadfdf-f0a9-4553-bbac-b1d12be690a1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:55.848778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:55.849733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:55.850014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:55.850708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] PATH_INFO: `/v3/users/86103d2ae8a542ee9207e46d292c3008` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:55.851638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=86103d2ae8a542ee9207e46d292c3008)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:55.853696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:56.051087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22d11d4bfe96474d94e90bc4cb5db80f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.052669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22d11d4bfe96474d94e90bc4cb5db80f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.053489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22d11d4bfe96474d94e90bc4cb5db80f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.061150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22d11d4bfe96474d94e90bc4cb5db80f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.080975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22d11d4bfe96474d94e90bc4cb5db80f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.211914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86103d2ae8a542ee9207e46d292c3008'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.212701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86103d2ae8a542ee9207e46d292c3008'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.229156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86103d2ae8a542ee9207e46d292c3008'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.229947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86103d2ae8a542ee9207e46d292c3008'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.238542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86103d2ae8a542ee9207e46d292c3008'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:56.248491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1145/2266] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:55 2022] DELETE /identity/v3/users/22d11d4bfe96474d94e90bc4cb5db80f => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:56.322144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:56.419855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1122/2267] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:55 2022] DELETE /identity/v3/users/86103d2ae8a542ee9207e46d292c3008 => generated 0 bytes in 656 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:57.213402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:57.214990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:57.285204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431b7c62-dc41-4dab-8586-77e2d0ea7110 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:57.345248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:57.346173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.346484 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.346830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] PATH_INFO: `/v3/projects/7f90e344247049fb84cb40ccf41ef7d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.348001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f90e344247049fb84cb40ccf41ef7d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:57.351062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:57.364428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:57.366944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:57.381871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28129775-2100-4633-96ac-92d8feb2c1ba admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:57.388249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:57.389277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.389566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.389981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] PATH_INFO: `/v3/projects/4fd162af736b4a5fb4367f4241d040ad` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.391232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4fd162af736b4a5fb4367f4241d040ad)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:57.393303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:57.489950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f90e344247049fb84cb40ccf41ef7d2', 'request_id': 'req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.495737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f90e344247049fb84cb40ccf41ef7d2', 'request_id': 'req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.596254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4fd162af736b4a5fb4367f4241d040ad', 'request_id': 'req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.602979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4fd162af736b4a5fb4367f4241d040ad', 'request_id': 'req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.609517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4fd162af736b4a5fb4367f4241d040ad', 'request_id': 'req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.623246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f90e344247049fb84cb40ccf41ef7d2', 'request_id': 'req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.637197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] The token cache is being invalidate because project 7f90e344247049fb84cb40ccf41ef7d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:57.637594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.638291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1eb91230-5d5e-4f89-8a65-c41cc86e90ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.640189 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1146/2268] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:57 2022] DELETE /identity/v3/projects/7f90e344247049fb84cb40ccf41ef7d2 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:57.647463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8d56-bede-4a4b-8c7f-cd0b619011e4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.647995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8d56-bede-4a4b-8c7f-cd0b619011e4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.648421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3b8d56-bede-4a4b-8c7f-cd0b619011e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.678910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ea3b8d56-bede-4a4b-8c7f-cd0b619011e4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:57.680259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ea3b8d56-bede-4a4b-8c7f-cd0b619011e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:57.750196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] The token cache is being invalidate because project 4fd162af736b4a5fb4367f4241d040ad was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:33:57.750196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.750196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1d9596ca-fde4-455d-8bb9-b0232bd5d0f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:57.750196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1123/2269] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:33:57 2022] DELETE /identity/v3/projects/4fd162af736b4a5fb4367f4241d040ad => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:57.756869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d002d-efb9-471a-910b-ca2d10b637a7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.757602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d002d-efb9-471a-910b-ca2d10b637a7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.758048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0d002d-efb9-471a-910b-ca2d10b637a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.781110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1147/2270] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:57.782385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5f0d002d-efb9-471a-910b-ca2d10b637a7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:57.784516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.784799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.785110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.786406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5f0d002d-efb9-471a-910b-ca2d10b637a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:57.801102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1124/2271] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:57.804212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.804495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.805375 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.811239 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:57.812709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:57.824798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1148/2272] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:57.829590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:57.829693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:57.830431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:57.838893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b830f6b-f49e-4882-99c5-5a53d2378bdf None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:57.839412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1125/2273] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:57.843560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:57.843963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:57.844598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.844882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.845202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.846238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:57.846581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:57.848242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:57.854111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305998a6-60a7-49a2-9436-d9537d00044b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:57.859155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:57.859770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.860080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.860352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.861000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:33:57.861440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1149/2274] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:57.861577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:57.862941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:57.866515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:57.876415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b634da4-6db0-4620-a7a3-542a85437ece admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:57.877175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1126/2275] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:33:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:33:57.882110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:57.886878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.886878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.886878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.887194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:57.887296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:57.887296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:57.894290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ba6fee-f649-4277-b617-04c6a5090012 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:57.899257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:57.899931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:57.900209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:57.900482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:57.901150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:57.903038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:57.991742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1150/2276] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/projects => generated 373 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:57.995955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.038291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1127/2277] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:58.041350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.070054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821e3a89-09c3-4ac0-8d95-6ebbd9329073 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.125765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31833f27-1ad2-4712-b7b7-c76320582336 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.131480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.141803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.141803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.141803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.141803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.141803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.144172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.148565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.149062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.149409 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.150194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.152280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.273312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1128/2278] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:58 2022] POST /identity/v3/users => generated 293 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:58.274191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1151/2279] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:33:57 2022] POST /identity/v3/users => generated 287 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:33:58.275584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.278927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.294487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd40856-920d-4e35-95f0-29d617459245 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.302841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.317764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.317764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.317764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.318455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51db10ea-2cd6-469b-8ade-b90eb25950fd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.318660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.318660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.318984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.318984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.318984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.318984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.319588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.319588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.329431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1129/2280] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:58.339894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.340639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1152/2281] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:58.348211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.355867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701a82eb-6b59-4d5b-8a61-d7d57920fea5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.361751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.362625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.363569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.364025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.364732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.366494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.375910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b032c8-5359-4d60-82ec-cc9c1ede23c5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.381233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:58.381634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1130/2282] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:58.386052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.387380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.388720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.389426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.390245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.392090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.397306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.400261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8edd70b0-2d78-4073-9be4-5b8e0f019b56 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.406418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262a063b-58b0-49cc-97e5-15b02d671e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.407157 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a063b-58b0-49cc-97e5-15b02d671e69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.412336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a063b-58b0-49cc-97e5-15b02d671e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.412336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262a063b-58b0-49cc-97e5-15b02d671e69 admin admin] PATH_INFO: `/v3/projects/323b32167cff4d289e23560d3daf9e06/users/3c820c1dad5e4a039f92b9f8b5d6b8aa/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.425304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a063b-58b0-49cc-97e5-15b02d671e69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=323b32167cff4d289e23560d3daf9e06, user_id=3c820c1dad5e4a039f92b9f8b5d6b8aa, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.428536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262a063b-58b0-49cc-97e5-15b02d671e69 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.430610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1153/2283] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:33:58 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:33:58.435780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.449525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500cd6e1-b46b-4b24-8a71-4d74368669e4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.452616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e28a83-40b0-4dcc-bac3-9640ff2c94ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.453419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e28a83-40b0-4dcc-bac3-9640ff2c94ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.453899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e28a83-40b0-4dcc-bac3-9640ff2c94ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.454246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e28a83-40b0-4dcc-bac3-9640ff2c94ee admin admin] PATH_INFO: `/v3/projects/fa04fb474091433bb6e4f1cd939934fa/users/da157a93c7d84b8d8b84755a47b686bb/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.475313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e28a83-40b0-4dcc-bac3-9640ff2c94ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa04fb474091433bb6e4f1cd939934fa, user_id=da157a93c7d84b8d8b84755a47b686bb, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.480041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e28a83-40b0-4dcc-bac3-9640ff2c94ee admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.572782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1131/2284] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:58 2022] PUT /identity/v3/projects/323b32167cff4d289e23560d3daf9e06/users/3c820c1dad5e4a039f92b9f8b5d6b8aa/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:58.584605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.584897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.585206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.622172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] MFA Rules not processed for user `3c820c1dad5e4a039f92b9f8b5d6b8aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:58.624031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1154/2285] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:33:58 2022] PUT /identity/v3/projects/fa04fb474091433bb6e4f1cd939934fa/users/da157a93c7d84b8d8b84755a47b686bb/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:33:58.630184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:58.633217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.634352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.634753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.663025 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:58.668401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:58.721349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1132/2286] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:58 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:33:58.735536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.738231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:58.791147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1155/2287] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:58.800927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.802626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:58.841196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d60c76-d96b-4ff4-a534-2b4bac89a10d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.855519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.856423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.856867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.857188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.873289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.874743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.886081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1133/2288] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:58 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:33:58.915031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:58.929881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc26872c-aaac-452d-9037-9152e2de4e01 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.960831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d0c621-9ce7-46c3-a109-33309ddc965e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:58.967275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdaa40c-b045-4483-973d-648904c45e2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.968216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaa40c-b045-4483-973d-648904c45e2b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.968589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaa40c-b045-4483-973d-648904c45e2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.968962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdaa40c-b045-4483-973d-648904c45e2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.972570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:58.973784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:58.974279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:58.974744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] PATH_INFO: `/v3/users/3c820c1dad5e4a039f92b9f8b5d6b8aa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:58.977315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c820c1dad5e4a039f92b9f8b5d6b8aa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:58.979358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:58.999028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdaa40c-b045-4483-973d-648904c45e2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:59.002049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdaa40c-b045-4483-973d-648904c45e2b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:59.024803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1156/2289] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:33:59.278782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:33:59.437702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c820c1dad5e4a039f92b9f8b5d6b8aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:59.438514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c820c1dad5e4a039f92b9f8b5d6b8aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:59.456324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c820c1dad5e4a039f92b9f8b5d6b8aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:59.459913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c820c1dad5e4a039f92b9f8b5d6b8aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:59.467939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c820c1dad5e4a039f92b9f8b5d6b8aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:33:59.573661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1134/2290] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:33:58 2022] DELETE /identity/v3/users/3c820c1dad5e4a039f92b9f8b5d6b8aa => generated 0 bytes in 659 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:33:59.578696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:59.579359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:59.579635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:59.603829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:33:59.604995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:59.681433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1157/2291] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:33:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:33:59.695423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:33:59.695715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:33:59.795179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339d2e5-c3c6-41c7-b621-55cd89310b75 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:33:59.798871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7b9dc4-fc87-42df-86bc-7e43a9033a9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:33:59.799641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b9dc4-fc87-42df-86bc-7e43a9033a9f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:33:59.799923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b9dc4-fc87-42df-86bc-7e43a9033a9f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:33:59.800193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7b9dc4-fc87-42df-86bc-7e43a9033a9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:33:59.818172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b9dc4-fc87-42df-86bc-7e43a9033a9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:33:59.823826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7b9dc4-fc87-42df-86bc-7e43a9033a9f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:33:59.830321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1135/2292] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:33:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:00.480911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:00.482434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:00.507031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7190873e-4784-4c8c-9e80-1c0b3eaafff3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:00.524967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:00.525660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:00.525937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:00.526214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] PATH_INFO: `/v3/projects/323b32167cff4d289e23560d3daf9e06` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:00.527874 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] RBAC: Authorizing `identity:delete_project(project_id=323b32167cff4d289e23560d3daf9e06)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:00.530198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:00.602147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:00.731632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '323b32167cff4d289e23560d3daf9e06', 'request_id': 'req-878153df-8467-4af2-b4be-e7f60c52379c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:00.740547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '323b32167cff4d289e23560d3daf9e06', 'request_id': 'req-878153df-8467-4af2-b4be-e7f60c52379c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:00.907014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '323b32167cff4d289e23560d3daf9e06', 'request_id': 'req-878153df-8467-4af2-b4be-e7f60c52379c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:00.931863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] The token cache is being invalidate because project 323b32167cff4d289e23560d3daf9e06 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:00.931863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:00.931863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-878153df-8467-4af2-b4be-e7f60c52379c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:00.931863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1158/2293] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:00 2022] DELETE /identity/v3/projects/323b32167cff4d289e23560d3daf9e06 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:00.937144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:00.937661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:00.938161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:00.980538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:00.981992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:01.125085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1136/2294] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:01.147412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:01.149004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:01.149004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:01.173615 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:01.178958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:01.189732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1159/2295] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:01.191298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:01.204780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845ce6a8-90a4-43a1-bd04-5c6b40ce8fff None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:01.218261 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:01.218886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:01.219616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:01.220095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:01.221320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:01.221688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:01.223420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:01.244077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1137/2296] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:01.248515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:01.262918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52455f55-acf4-4c25-b5a0-fc81cf18481c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:01.269178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:01.270982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:01.271729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:01.272074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:01.276692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:01.278068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:01.462055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:01.462359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1160/2297] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:01 2022] POST /identity/v3/projects => generated 371 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:01.493581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68f29cc-a940-4320-978d-630f783959a2 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:01.580677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:01.706948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:01.708208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:01.711450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:01.714096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:01.714871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:01.719504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:01.845860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1138/2298] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:01 2022] POST /identity/v3/users => generated 286 bytes in 383 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:01.851814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:01.863292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ac77bd-2396-4db3-85b2-fc942b1cfe19 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:01.881066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:01.881729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:01.882637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:01.882637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:01.882994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:01.884703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:01.929341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:01.941437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1161/2299] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:01.948166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47c0797-6842-4d67-8b22-8998815f3821 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:01.969860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc1edde-45b1-41d6-bd58-10634547f634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:01.970810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1edde-45b1-41d6-bd58-10634547f634 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:01.971015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1edde-45b1-41d6-bd58-10634547f634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:01.971513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc1edde-45b1-41d6-bd58-10634547f634 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:01.974822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1edde-45b1-41d6-bd58-10634547f634 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:01.980517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc1edde-45b1-41d6-bd58-10634547f634 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:02.009710 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1139/2300] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:01 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:02.013399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:02.027018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b34a49-0d8f-40c0-a422-667aca1eefaa admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:02.044616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7872c6a-e9c2-42de-9671-88750c714721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:02.045678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7872c6a-e9c2-42de-9671-88750c714721 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.045967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7872c6a-e9c2-42de-9671-88750c714721 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.046243 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7872c6a-e9c2-42de-9671-88750c714721 admin admin] PATH_INFO: `/v3/projects/d7b0f71800734a43a5d955e666ae7c1c/users/122d2cfe67eb4c46ab2dcc24f2edf041/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.080676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7872c6a-e9c2-42de-9671-88750c714721 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7b0f71800734a43a5d955e666ae7c1c, user_id=122d2cfe67eb4c46ab2dcc24f2edf041, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:02.084350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7872c6a-e9c2-42de-9671-88750c714721 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:02.120667 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f73d5e-6802-4751-8c1c-a6544a0f8a79 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.120949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f73d5e-6802-4751-8c1c-a6544a0f8a79 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.121568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f73d5e-6802-4751-8c1c-a6544a0f8a79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.177866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-98f73d5e-6802-4751-8c1c-a6544a0f8a79 None None] MFA Rules not processed for user `da157a93c7d84b8d8b84755a47b686bb`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:02.179328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-98f73d5e-6802-4751-8c1c-a6544a0f8a79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:02.282988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1162/2301] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:02 2022] PUT /identity/v3/projects/d7b0f71800734a43a5d955e666ae7c1c/users/122d2cfe67eb4c46ab2dcc24f2edf041/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:02.287459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.288981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.289466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.320869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1140/2302] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:02.323312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.324013 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.324513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.354601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:02.355868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:02.380123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] MFA Rules not processed for user `122d2cfe67eb4c46ab2dcc24f2edf041`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:02.381130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:02.550427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1163/2303] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:02.560494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:02.569151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1141/2304] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:02 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:02.569151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:02.576987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:02.577138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:02.580314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:02.705712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223de08f-1dea-49ea-a59b-98b5345d818b None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:02.711227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:02.713064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.713530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.714002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.725112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:02.726314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db56ddc5-0dcf-4ef9-a9b8-5365f26d94bd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:02.729418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:02.732641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:02.734121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.734582 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.735144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.736439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:02.739883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:02.747357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1142/2305] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:02 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:02.748477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:02.750074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:02.851433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1164/2306] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:02 2022] POST /identity/v3/projects => generated 371 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:02.852363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:02.861601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7453f11a-5e14-4177-a518-e7e935d8570e service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:02.993544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:02.993544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:02.993544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:02.993544 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:02.996623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0a1168-b489-4a80-b9a3-d2948ffb160e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.009197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.010315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.010831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.011334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] PATH_INFO: `/v3/users/122d2cfe67eb4c46ab2dcc24f2edf041` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.013378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=122d2cfe67eb4c46ab2dcc24f2edf041)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.017291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.089210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.091076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.111114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1143/2307] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:34:03.111114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:03.140306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d4457-cf70-425f-bc84-b5c54c47d04a service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.157516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.158156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.158438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.158747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.159404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.162194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.344803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1144/2308] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:03 2022] POST /identity/v3/users => generated 292 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:03.350796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:03.361738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e961ed1b-f57d-46e4-971d-82cc0c8c1185 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.368139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.368739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.369029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.369326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.369981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.371748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.387446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1145/2309] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:03.392823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:03.406267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed1ff1-0d1e-40f1-8cbf-ebc7aa71a701 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.414234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.414911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.415313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.415598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] PATH_INFO: `/v3/projects/06da8cc624c54a3b99cc8880bcda7a31/users/c7455dbd9f98408ea6d923ba32c916b4/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.422604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '122d2cfe67eb4c46ab2dcc24f2edf041'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:03.423607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '122d2cfe67eb4c46ab2dcc24f2edf041'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:03.424233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '122d2cfe67eb4c46ab2dcc24f2edf041'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:03.432706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '122d2cfe67eb4c46ab2dcc24f2edf041'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:03.434790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] RBAC: Authorizing `identity:create_grant(project_id=06da8cc624c54a3b99cc8880bcda7a31, user_id=c7455dbd9f98408ea6d923ba32c916b4, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.438276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.446087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-2ae6eb00-0cc2-459b-bc8b-d7951190bca9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '122d2cfe67eb4c46ab2dcc24f2edf041'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:03.616147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1165/2310] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:02 2022] DELETE /identity/v3/users/122d2cfe67eb4c46ab2dcc24f2edf041 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:03.616582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1146/2311] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:03 2022] PUT /identity/v3/projects/06da8cc624c54a3b99cc8880bcda7a31/users/c7455dbd9f98408ea6d923ba32c916b4/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:03.620363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.620667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.621100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.622284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:03.625487 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:03.646113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:03.647460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:03.832600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3ac3c2-e90a-41d3-9d67-80d187735adb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.836800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1166/2312] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:03.836800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:03.836800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:03.842771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.842771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.842771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.842771 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.843937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.843937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.850473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1147/2313] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:03 2022] GET /identity/v3/roles => generated 1420 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:03.854330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:03.928224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b4468d-8485-490e-a68a-772e92407a16 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.928224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804a9834-eb34-44a4-8591-69c48bd7817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.928224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9834-eb34-44a4-8591-69c48bd7817e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.928224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9834-eb34-44a4-8591-69c48bd7817e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.929652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804a9834-eb34-44a4-8591-69c48bd7817e admin admin] PATH_INFO: `/v3/projects/06da8cc624c54a3b99cc8880bcda7a31/users/c7455dbd9f98408ea6d923ba32c916b4/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.929652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9834-eb34-44a4-8591-69c48bd7817e admin admin] RBAC: Authorizing `identity:create_grant(project_id=06da8cc624c54a3b99cc8880bcda7a31, user_id=c7455dbd9f98408ea6d923ba32c916b4, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.929652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804a9834-eb34-44a4-8591-69c48bd7817e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.930118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec9983-8d1a-4fd6-aece-032f96639b86 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:03.930118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:03.930118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:03.930118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:03.930486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:03.930579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1148/2314] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:03 2022] PUT /identity/v3/projects/06da8cc624c54a3b99cc8880bcda7a31/users/c7455dbd9f98408ea6d923ba32c916b4/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:03.930666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:03.930666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:03.940457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1167/2315] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:04.572273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:04.572273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:04.572273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:04.597829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:04.602693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:04.695912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1149/2316] 10.222.0.73 () {58 vars in 967 bytes} [Tue Aug 23 22:34:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:04.711798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:04.713944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:04.809226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0207fbe6-4ec9-4039-b182-69be7c966fd2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:04.868092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:04.868092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:04.900599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32dfaa-7ce2-4b75-a45e-68337dae5b8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:04.901360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32dfaa-7ce2-4b75-a45e-68337dae5b8e service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:04.901704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32dfaa-7ce2-4b75-a45e-68337dae5b8e service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:04.902041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32dfaa-7ce2-4b75-a45e-68337dae5b8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:04.911246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32dfaa-7ce2-4b75-a45e-68337dae5b8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:04.913270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32dfaa-7ce2-4b75-a45e-68337dae5b8e service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:04.934264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1168/2317] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:34:05.024248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1c8825-77fb-4287-a585-a83d1d1f0423 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:05.030125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:05.030804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:05.031142 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:05.031418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] PATH_INFO: `/v3/projects/d7b0f71800734a43a5d955e666ae7c1c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:05.032575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7b0f71800734a43a5d955e666ae7c1c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:05.035405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:05.225806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7b0f71800734a43a5d955e666ae7c1c', 'request_id': 'req-e0353776-7e4f-4ab1-9a21-4f2a40de4971'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:05.231664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7b0f71800734a43a5d955e666ae7c1c', 'request_id': 'req-e0353776-7e4f-4ab1-9a21-4f2a40de4971'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:05.238140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7b0f71800734a43a5d955e666ae7c1c', 'request_id': 'req-e0353776-7e4f-4ab1-9a21-4f2a40de4971'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:05.397896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] The token cache is being invalidate because project d7b0f71800734a43a5d955e666ae7c1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:05.398179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:05.398698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0353776-7e4f-4ab1-9a21-4f2a40de4971 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:05.400602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1150/2318] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:04 2022] DELETE /identity/v3/projects/d7b0f71800734a43a5d955e666ae7c1c => generated 0 bytes in 541 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:05.405465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:05.415799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:05.415799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:05.428671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:05.429949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:05.509451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1169/2319] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:05.512363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:05.512880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:05.513420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:05.561463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:05.563086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:05.589386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1151/2320] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:05.595010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:05.615263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a883419a-b737-4376-8013-87f5e1335f55 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:05.621360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:05.622230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:05.623275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:05.623275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:05.623438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:05.623801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:05.625813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:05.651632 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1170/2321] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:05.651757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:05.672999 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df650d21-76d5-43be-abe2-76364afb7e87 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:05.683513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f255c0b-ebe6-4e53-9a6b-aa92399f20fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:05.684882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f255c0b-ebe6-4e53-9a6b-aa92399f20fb admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:05.685743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f255c0b-ebe6-4e53-9a6b-aa92399f20fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:05.685911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f255c0b-ebe6-4e53-9a6b-aa92399f20fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:05.687161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f255c0b-ebe6-4e53-9a6b-aa92399f20fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:05.690350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f255c0b-ebe6-4e53-9a6b-aa92399f20fb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:05.829786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:05.830766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1152/2322] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:05 2022] POST /identity/v3/projects => generated 361 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:05.849761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4779d138-517a-4bdf-9460-6c0ea61df7cc admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:05.951746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:05.953319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:05.953851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:05.954306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:05.955148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:05.958456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.026585 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.027414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.027414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.060221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] MFA Rules not processed for user `c7455dbd9f98408ea6d923ba32c916b4`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:06.062299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.087423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1171/2323] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:05 2022] POST /identity/v3/users => generated 281 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:06.096650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.119013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4eaf920-4054-42c2-81cb-22131787d4c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.124061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2840175e-d95d-4526-ad30-b17403300d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.125399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840175e-d95d-4526-ad30-b17403300d81 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.126032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840175e-d95d-4526-ad30-b17403300d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.126583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2840175e-d95d-4526-ad30-b17403300d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.127763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840175e-d95d-4526-ad30-b17403300d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.130852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2840175e-d95d-4526-ad30-b17403300d81 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.145440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1172/2324] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:06.146789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:06.148848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.149331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.149756 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.173561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:06.175033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.177801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1153/2325] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:06.178804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.193564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbab5022-fc53-4771-b756-c1e366311c89 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.201912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.202128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.202361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.202589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.203851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.205811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.222146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1154/2326] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:06 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:06.222146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.224956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.242811 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1173/2327] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:06.243099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.256229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae11f1cf-65b0-4523-80e8-69c35c1bc8fd None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.263593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.264435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.265594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.266256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] PATH_INFO: `/v3/projects/f7938cf7f00d442ea088e6edd065aa00/users/65217f3f093b4a10b715602f6f952c9d/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.283222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7938cf7f00d442ea088e6edd065aa00, user_id=65217f3f093b4a10b715602f6f952c9d, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.287403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.293669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961a31bf-fd31-4b7b-a93d-d19007dbaab8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.297576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.298197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.298479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.298754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.300649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.369945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.372384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6272160f-71ff-4894-a05f-e921f4ca06ec service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.381469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1174/2328] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:06 2022] PUT /identity/v3/projects/f7938cf7f00d442ea088e6edd065aa00/users/65217f3f093b4a10b715602f6f952c9d/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:06.382572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.384270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.392158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1155/2329] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:06.396231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.396516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.399663 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.457732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] MFA Rules not processed for user `65217f3f093b4a10b715602f6f952c9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:06.460257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.513868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580f0751-92fd-40db-a33b-933aafdac76c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.521595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.522719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.522989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.523403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.576218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1156/2330] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:06 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:06.615942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.617958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.630933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:06.631314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.631314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1175/2331] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:34:06.682775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5574a6-0903-4a6e-b345-25f045c4dba1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.686552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98214229-7688-4977-8087-5cb96d98dcb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.687169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214229-7688-4977-8087-5cb96d98dcb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.687451 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214229-7688-4977-8087-5cb96d98dcb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.687725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98214229-7688-4977-8087-5cb96d98dcb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.697245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98214229-7688-4977-8087-5cb96d98dcb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.703618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98214229-7688-4977-8087-5cb96d98dcb7 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:06.710442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1157/2332] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:06.726894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:06.792551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192d562e-e78c-4646-8dba-61e845698935 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:06.797499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:06.798139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:06.798421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:06.798716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] PATH_INFO: `/v3/users/65217f3f093b4a10b715602f6f952c9d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:06.799852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65217f3f093b4a10b715602f6f952c9d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:06.801898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:07.157939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65217f3f093b4a10b715602f6f952c9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:07.159169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65217f3f093b4a10b715602f6f952c9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:07.159871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65217f3f093b4a10b715602f6f952c9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:07.179869 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65217f3f093b4a10b715602f6f952c9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:07.197825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65217f3f093b4a10b715602f6f952c9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:07.299348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1176/2333] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:06 2022] DELETE /identity/v3/users/65217f3f093b4a10b715602f6f952c9d => generated 0 bytes in 573 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:07.304942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9899a77-b505-4dac-8e43-e54844e4531a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:07.305565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9899a77-b505-4dac-8e43-e54844e4531a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:07.305565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9899a77-b505-4dac-8e43-e54844e4531a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:07.330742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d9899a77-b505-4dac-8e43-e54844e4531a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:07.332952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d9899a77-b505-4dac-8e43-e54844e4531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:07.415071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1158/2334] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:07.427124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:07.428975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:07.467645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:07.518017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eede9b2b-20ca-411e-ac48-e105fc7a26f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:07.528867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:07.530576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:07.531181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:07.531790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:07.551413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:07.571183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:07.573868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1177/2335] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:07.597028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:07.597324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:07.598465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:07.623276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:07.624591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:07.725190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1159/2336] 10.222.0.73 () {58 vars in 967 bytes} [Tue Aug 23 22:34:07 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:07.739696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:07.741359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:07.850026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c19e56-3ff2-4d6d-b090-adc84b6f382b service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:07.859181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff5127c-53a1-465d-b232-2874a85b44eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:07.859763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5127c-53a1-465d-b232-2874a85b44eb service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:07.860144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5127c-53a1-465d-b232-2874a85b44eb service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:07.861236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5127c-53a1-465d-b232-2874a85b44eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:07.878031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5127c-53a1-465d-b232-2874a85b44eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:07.878031 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff5127c-53a1-465d-b232-2874a85b44eb service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:07.890020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1178/2337] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:34:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 23 22:34:08.152522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:08.154201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:08.168234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01020c50-6137-4ca9-a4a7-8252a429f0bb None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:08.173592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:08.174269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:08.174552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:08.175332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] PATH_INFO: `/v3/projects/f7938cf7f00d442ea088e6edd065aa00` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:08.176530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7938cf7f00d442ea088e6edd065aa00)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:08.178495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:08.351541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7938cf7f00d442ea088e6edd065aa00', 'request_id': 'req-c317a646-b025-4311-92ea-ae04ce51c6e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:08.358814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7938cf7f00d442ea088e6edd065aa00', 'request_id': 'req-c317a646-b025-4311-92ea-ae04ce51c6e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:08.368952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7938cf7f00d442ea088e6edd065aa00', 'request_id': 'req-c317a646-b025-4311-92ea-ae04ce51c6e2'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:08.464162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] The token cache is being invalidate because project f7938cf7f00d442ea088e6edd065aa00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:08.470068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:08.470068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c317a646-b025-4311-92ea-ae04ce51c6e2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:08.470068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1160/2338] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:08 2022] DELETE /identity/v3/projects/f7938cf7f00d442ea088e6edd065aa00 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:08.475123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:08.476273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:08.476965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:08.492989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:08.526046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:08.528264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:08.680636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1179/2339] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:08.693779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:08.694712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:08.695403 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:08.731631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:08.733884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:08.759780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1161/2340] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:08.764185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:08.787759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf3213f-072a-44c8-9e71-7024f60c53ed None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:08.805489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:08.809614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:08.809908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:08.810199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:08.811135 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:08.815915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:08.823398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:08.852423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1180/2341] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:08.860847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:08.887633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f62c1d-72bf-4c47-9460-17401b9300f8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:08.916961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:08.919889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:08.919889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:08.919889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:08.920553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:08.924178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:09.229525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1162/2342] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:08 2022] POST /identity/v3/projects => generated 371 bytes in 370 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:09.233435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:09.245899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750daacd-3789-46d2-94cf-47150c75d729 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:09.382081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:09.383257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:09.384335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:09.384920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:09.391751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:09.391751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:09.495000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1181/2343] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:09 2022] POST /identity/v3/users => generated 286 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:09.500324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:09.518300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9febb8-0e4d-4048-b99d-ed4199ea797a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:09.520020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:09.520456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:09.520992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:09.521639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:09.522234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:09.524129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:09.535723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1163/2344] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:09.540321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:09.556799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59fc7c9-7a5e-4538-b3a9-6fdefcfa30f0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:09.561437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af67d55c-755b-49cd-95b0-177158abd757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:09.562091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67d55c-755b-49cd-95b0-177158abd757 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:09.562371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67d55c-755b-49cd-95b0-177158abd757 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:09.562645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af67d55c-755b-49cd-95b0-177158abd757 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:09.563408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67d55c-755b-49cd-95b0-177158abd757 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:09.565433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af67d55c-755b-49cd-95b0-177158abd757 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:09.584519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1182/2345] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:09 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:09.585963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:09.598179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c29e8b-b958-4829-9bef-48a6ce86f133 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:09.606409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:09.607069 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:09.607650 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:09.607792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] PATH_INFO: `/v3/projects/0fb8b4774d54411099bec473966aced5/users/e037a0abf77f45d7befabbf8d05fcd53/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:09.628449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0fb8b4774d54411099bec473966aced5, user_id=e037a0abf77f45d7befabbf8d05fcd53, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:09.636020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:09.789736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1164/2346] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:09 2022] PUT /identity/v3/projects/0fb8b4774d54411099bec473966aced5/users/e037a0abf77f45d7befabbf8d05fcd53/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:09.799351 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:09.799660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:09.799960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:09.823672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:09.824901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:09.894487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1183/2347] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:09.905633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:09.910176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:09.970407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1afb2bd-64bd-44e5-9b68-28b7ae9327f8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:09.974413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:09.975027 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:09.975344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:09.975622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:09.986226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:09.987653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:09.998263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1165/2348] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:10.251691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:10.859893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:10.861356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:10.927258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497285b4-c9c7-4042-9547-c38746f8e1ab None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:10.931202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:10.931853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:10.932157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:10.932446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:10.934300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:11.014530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:11.016489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1bf24d-8596-4ce9-8ba1-d16fb7269c80 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:11.027675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1184/2349] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:10 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:11.104274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:11.178138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:11.179613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:11.192721 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99b2979-ca30-46cd-8c49-212784742b73 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:11.200978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:11.201665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:11.201992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:11.202201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] PATH_INFO: `/v3/users/55253e6867224ed78cca27eab3a40b68` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:11.203371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] RBAC: Authorizing `identity:delete_user(user_id=55253e6867224ed78cca27eab3a40b68)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:11.205290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:11.490916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55253e6867224ed78cca27eab3a40b68'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:11.491869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55253e6867224ed78cca27eab3a40b68'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:11.505774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55253e6867224ed78cca27eab3a40b68'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:11.506546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55253e6867224ed78cca27eab3a40b68'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:11.513859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-3746293d-812c-406e-91bc-0b9bba1fbbde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55253e6867224ed78cca27eab3a40b68'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:11.533852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:11.536104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:11.536104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:11.575121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] MFA Rules not processed for user `e037a0abf77f45d7befabbf8d05fcd53`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:11.575534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:11.613063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1166/2350] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:11 2022] DELETE /identity/v3/users/55253e6867224ed78cca27eab3a40b68 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:11.617638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:11.618166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:11.618646 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:11.641178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:11.642831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:11.656370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1185/2351] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:11 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:11.668545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:11.669970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:11.685059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5e8664-ec56-439e-99a0-a46fd8dfefd2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:11.721476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1167/2352] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:11.732815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:11.733525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:11.739773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:11.740413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:11.740695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:11.740968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:11.799603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:11.799849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f045b9e8-471d-4a4c-9ea9-4f1c34255abf None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:11.801199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:11.804075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46818764-34e4-4a82-8823-7cce80ca637d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:11.804547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46818764-34e4-4a82-8823-7cce80ca637d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:11.804820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46818764-34e4-4a82-8823-7cce80ca637d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:11.805189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46818764-34e4-4a82-8823-7cce80ca637d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:11.810875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1186/2353] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:11 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:11.814770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46818764-34e4-4a82-8823-7cce80ca637d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:11.816330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46818764-34e4-4a82-8823-7cce80ca637d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:11.827921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1168/2354] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:12.086906 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:12.088388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:12.103019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0eba336-a979-44b5-85de-ade3ff4e3d91 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:12.107801 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:12.108444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:12.108723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:12.109143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] PATH_INFO: `/v3/projects/7aa16e85a90f44c4872c92d5db467f3e` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:12.110175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7aa16e85a90f44c4872c92d5db467f3e)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:12.115012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:12.256190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7aa16e85a90f44c4872c92d5db467f3e', 'request_id': 'req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:12.263060 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7aa16e85a90f44c4872c92d5db467f3e', 'request_id': 'req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:12.367878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7aa16e85a90f44c4872c92d5db467f3e', 'request_id': 'req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:12.381676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] The token cache is being invalidate because project 7aa16e85a90f44c4872c92d5db467f3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:12.382078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:12.382765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f4b2db0f-1aa2-4e8b-80b3-57cc6692e294 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:12.384383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1187/2355] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:12 2022] DELETE /identity/v3/projects/7aa16e85a90f44c4872c92d5db467f3e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:12.391957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:12.396110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:12.396110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:12.423231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:12.424825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:12.511314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1169/2356] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:12.512363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:12.512759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:12.513149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:12.540882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:12.542495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:12.552798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1188/2357] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:12.556435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:12.567549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7486f07-7683-4536-9191-2b31e71597a5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:12.572463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:12.573227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:12.573416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:12.573660 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:12.574320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:12.574625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:12.576304 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:12.593645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1170/2358] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:12.598252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:12.608999 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e755b7f4-369e-4598-9edf-336231accdd0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:12.615548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:12.616203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:12.616486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:12.616795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:12.617467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:12.619695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:12.673877 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:12.774794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1189/2359] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:12 2022] POST /identity/v3/projects => generated 383 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:12.780594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:12.809533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b855dc71-ea2e-40c1-88eb-016fb09a7a0b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:12.888784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:12.889565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:12.889890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:12.890206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:12.890894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:12.893128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:13.040943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1171/2360] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:12 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:13.045440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:13.058173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d78b59-40aa-4372-b405-ffce6365dc22 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:13.062856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:13.063544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:13.063883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:13.064205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:13.068206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:13.068206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:13.084802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1190/2361] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:13.087704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:13.098424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9627b3a9-a285-41ed-b6d4-0b7e5b0e5102 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:13.103355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:13.103977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:13.104259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:13.104574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:13.105265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:13.106945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:13.118815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1172/2362] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:13.123102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:13.142131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f165d3b7-b0cb-4ddf-acf3-633d45818b5f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:13.142131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae2d0b-9897-420f-90d9-3b869936479d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:13.142131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae2d0b-9897-420f-90d9-3b869936479d admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:13.142131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae2d0b-9897-420f-90d9-3b869936479d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:13.142553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae2d0b-9897-420f-90d9-3b869936479d admin admin] PATH_INFO: `/v3/projects/182b0a6a36fb4c14816d9f94b270686c/users/adc12894a7c94ae1b06340353fd99654/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:13.157437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae2d0b-9897-420f-90d9-3b869936479d admin admin] RBAC: Authorizing `identity:create_grant(project_id=182b0a6a36fb4c14816d9f94b270686c, user_id=adc12894a7c94ae1b06340353fd99654, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:13.160284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae2d0b-9897-420f-90d9-3b869936479d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:13.219340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:13.221440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:13.269147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1191/2363] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:13 2022] PUT /identity/v3/projects/182b0a6a36fb4c14816d9f94b270686c/users/adc12894a7c94ae1b06340353fd99654/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:13.275406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:13.276126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:13.276782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:13.320300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64f9f9d-2991-43d7-9746-65001e50dddd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:13.321894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:13.325329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:13.408557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:13.409362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:13.409702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:13.410081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:13.412140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1192/2364] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:13.414661 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:13.418776 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:13.420220 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:13.422432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:13.431446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:13.434335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e47ce29-1c3a-43d4-a236-b8eab3982d90 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:13.446872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1173/2365] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:13.491079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9af2d7e-88f3-40f4-acbf-4eaa06506ba9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:13.494835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:13.495497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:13.495786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:13.496063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:13.506430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:13.507957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:13.518005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1193/2366] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:34:14.664251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:14.720485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dbc5d-4b86-4916-96a3-0b6dee224087 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:14.720767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dbc5d-4b86-4916-96a3-0b6dee224087 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:14.721266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857dbc5d-4b86-4916-96a3-0b6dee224087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:14.742329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-857dbc5d-4b86-4916-96a3-0b6dee224087 None None] MFA Rules not processed for user `e037a0abf77f45d7befabbf8d05fcd53`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:14.743805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-857dbc5d-4b86-4916-96a3-0b6dee224087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:14.840515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1174/2367] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:14 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:15.330460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:15.333042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.347169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5282a65-8d79-4cdb-af4f-8aab34ad6aba service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:15.352840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:15.353619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.353971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.354308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] PATH_INFO: `/v3/users/6dfc723a05e74342ab277d77d02317c1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.355552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6dfc723a05e74342ab277d77d02317c1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:15.357612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:15.401310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:15.529815 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.530103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.530381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.567006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] MFA Rules not processed for user `adc12894a7c94ae1b06340353fd99654`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:15.568863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.650444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6dfc723a05e74342ab277d77d02317c1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.651417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6dfc723a05e74342ab277d77d02317c1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.652922 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6dfc723a05e74342ab277d77d02317c1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.660910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6dfc723a05e74342ab277d77d02317c1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.665387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1175/2368] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:15.674867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:15.678727 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.686655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6dfc723a05e74342ab277d77d02317c1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:15.782211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e5f92-dcf3-4455-8ac3-260dcb490613 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:15.787570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:15.788895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.788895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.789281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.791716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.806210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:15.807759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bac46c8-ccc4-49dd-a901-28aee37ea5c1 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:15.819587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1176/2369] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:15.848344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1194/2370] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:15 2022] DELETE /identity/v3/users/6dfc723a05e74342ab277d77d02317c1 => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:15.853314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:15.853627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:15.853933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:15.888703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:15.890463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:15.959795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1177/2371] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:15.971095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:15.973131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:16.052271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af624ae0-5206-4638-b5bf-777d951e6ac1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:16.055732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:16.056705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:16.057081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:16.057364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:16.068301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:16.070895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:16.097186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1195/2372] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:16.827959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:16.830126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:16.844032 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370d06d2-700b-4b2e-ae7e-4db5d53fe177 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:16.849379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:16.850228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:16.850578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:16.851068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] PATH_INFO: `/v3/projects/82385f33e9134e00a9984f7e83390ba8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:16.852295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=82385f33e9134e00a9984f7e83390ba8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:16.854351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:16.981973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:17.093160 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82385f33e9134e00a9984f7e83390ba8', 'request_id': 'req-adb1bdf1-0337-45aa-a933-aed5a902ce07'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.101739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82385f33e9134e00a9984f7e83390ba8', 'request_id': 'req-adb1bdf1-0337-45aa-a933-aed5a902ce07'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.110051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82385f33e9134e00a9984f7e83390ba8', 'request_id': 'req-adb1bdf1-0337-45aa-a933-aed5a902ce07'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.219087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] The token cache is being invalidate because project 82385f33e9134e00a9984f7e83390ba8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:17.219385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.220322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-adb1bdf1-0337-45aa-a933-aed5a902ce07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:17.221898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1178/2373] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:16 2022] DELETE /identity/v3/projects/82385f33e9134e00a9984f7e83390ba8 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:17.673847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:17.806186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:17.807723 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:17.820911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e52bc2-0f3c-47b7-b881-72bee305b6c8 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:17.882727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:17.883603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:17.883889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:17.884167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] PATH_INFO: `/v3/users/423fdd722f384754a0821ac491ae7856` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:17.885367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] RBAC: Authorizing `identity:delete_user(user_id=423fdd722f384754a0821ac491ae7856)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:17.887331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.215034 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '423fdd722f384754a0821ac491ae7856'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:18.215683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '423fdd722f384754a0821ac491ae7856'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:18.216340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '423fdd722f384754a0821ac491ae7856'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:18.226269 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '423fdd722f384754a0821ac491ae7856'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:18.239957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '423fdd722f384754a0821ac491ae7856'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:18.327215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1196/2374] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:17 2022] DELETE /identity/v3/users/423fdd722f384754a0821ac491ae7856 => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:18.331632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168337-2563-48bd-8870-31e402f8a638 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.332253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168337-2563-48bd-8870-31e402f8a638 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.332253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c168337-2563-48bd-8870-31e402f8a638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.353313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2c168337-2563-48bd-8870-31e402f8a638 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:18.354531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2c168337-2563-48bd-8870-31e402f8a638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:18.414889 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1179/2375] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:34:18.424707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:18.428364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:18.497987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70952ed2-dccc-44e4-a4a1-6656160ce749 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:18.511926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:18.512950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:18.513183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:18.513458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:18.513707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:18.526147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:18.527796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:18.540020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1197/2376] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:19.139215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:19.153515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526068c4-ccfc-4ce9-84d6-7fe09a7392fb None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.153842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526068c4-ccfc-4ce9-84d6-7fe09a7392fb None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.154123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526068c4-ccfc-4ce9-84d6-7fe09a7392fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.174542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-526068c4-ccfc-4ce9-84d6-7fe09a7392fb None None] MFA Rules not processed for user `adc12894a7c94ae1b06340353fd99654`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:19.175844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-526068c4-ccfc-4ce9-84d6-7fe09a7392fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:19.234574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:19.239239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1180/2377] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:19.372862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.430098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:19.430098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69beb0ae-01df-4d0a-a38b-50a8725b4111 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.430098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.430623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.430623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.430623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] PATH_INFO: `/v3/projects/c5a8f4534c364a8fb272bf37487a41e6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.430623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5a8f4534c364a8fb272bf37487a41e6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.430623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.577099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5a8f4534c364a8fb272bf37487a41e6', 'request_id': 'req-ef44b36b-78a8-410d-a656-32c9b66021c3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:19.588668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5a8f4534c364a8fb272bf37487a41e6', 'request_id': 'req-ef44b36b-78a8-410d-a656-32c9b66021c3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:19.703633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5a8f4534c364a8fb272bf37487a41e6', 'request_id': 'req-ef44b36b-78a8-410d-a656-32c9b66021c3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:19.719434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] The token cache is being invalidate because project c5a8f4534c364a8fb272bf37487a41e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:19.720279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:19.721292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef44b36b-78a8-410d-a656-32c9b66021c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:19.723658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1198/2378] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:19 2022] DELETE /identity/v3/projects/c5a8f4534c364a8fb272bf37487a41e6 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:19.728859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.729370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.729505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.753366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:19.755738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:19.839141 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1181/2379] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:19.847998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.848464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.848778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.872350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:19.873942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:19.884393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1199/2380] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:19.888665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.901516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37016b12-1124-4f4a-ab75-10464138fd29 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.907762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.908407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.908878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.909194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.911400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:19.911708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.913627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:19.933088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1182/2381] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:19.935002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:19.945644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c31412-f8f7-4eb0-b5b3-d1c4cc72dad9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:19.950718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:19.951905 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:19.952166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:19.952436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:19.953111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:19.955386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.111799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1200/2382] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:19 2022] POST /identity/v3/projects => generated 373 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:20.118535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.129216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d53425-a1ff-4149-9cb0-f699245b1951 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:20.214039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:20.214731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.215193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.215674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.216559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:20.218859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.392572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1183/2383] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:20 2022] POST /identity/v3/users => generated 287 bytes in 280 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:20.397809 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.412959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaa38f6-805a-4f40-80f6-a326ba6d671d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:20.421696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:20.423191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.424236 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.424894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.426193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:20.428213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.445178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1201/2384] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:20.446081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.466989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fc9983-26ac-465f-a833-970e7d6d9773 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:20.475986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285861b3-b70a-489b-810e-7b29694f7347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:20.477520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285861b3-b70a-489b-810e-7b29694f7347 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.477520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285861b3-b70a-489b-810e-7b29694f7347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.477986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285861b3-b70a-489b-810e-7b29694f7347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.478762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285861b3-b70a-489b-810e-7b29694f7347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:20.480594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285861b3-b70a-489b-810e-7b29694f7347 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.505357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.512558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1184/2385] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:20.516522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445aefcf-f06d-4a0b-ba98-92277a5343f0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:20.528110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:20.529414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.529889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.530623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] PATH_INFO: `/v3/projects/0ecb3ee1dceb40f68765b515234f14b6/users/eea6d347d99b4765a6d3388db431b16a/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.556971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ecb3ee1dceb40f68765b515234f14b6, user_id=eea6d347d99b4765a6d3388db431b16a, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:20.559742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:20.755720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1202/2386] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:20 2022] PUT /identity/v3/projects/0ecb3ee1dceb40f68765b515234f14b6/users/eea6d347d99b4765a6d3388db431b16a/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:20.768231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7538f87-a573-43a9-9029-345516554f5a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:20.768769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7538f87-a573-43a9-9029-345516554f5a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:20.769322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7538f87-a573-43a9-9029-345516554f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:20.809342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b7538f87-a573-43a9-9029-345516554f5a None None] MFA Rules not processed for user `eea6d347d99b4765a6d3388db431b16a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:20.810986 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b7538f87-a573-43a9-9029-345516554f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:20.907177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1185/2387] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:20.920868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:20.923344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:21.010668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac35d4c9-45aa-41a3-995c-f42433a6db68 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:21.014674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4e0fb2-8b11-4ce8-a664-dc4d9d77cf45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:21.016134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e0fb2-8b11-4ce8-a664-dc4d9d77cf45 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:21.016318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e0fb2-8b11-4ce8-a664-dc4d9d77cf45 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:21.016597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4e0fb2-8b11-4ce8-a664-dc4d9d77cf45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:21.027437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e0fb2-8b11-4ce8-a664-dc4d9d77cf45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:21.029309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4e0fb2-8b11-4ce8-a664-dc4d9d77cf45 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:21.043175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1203/2388] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:20 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:21.074542 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7538f87-a573-43a9-9029-345516554f5a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:21.178539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7538f87-a573-43a9-9029-345516554f5a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:21.183631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:21.184328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:21.184609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:21.184903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] PATH_INFO: `/v3/users/eea6d347d99b4765a6d3388db431b16a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:21.186349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] RBAC: Authorizing `identity:delete_user(user_id=eea6d347d99b4765a6d3388db431b16a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:21.188401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:21.508887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eea6d347d99b4765a6d3388db431b16a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.510275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eea6d347d99b4765a6d3388db431b16a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.534773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eea6d347d99b4765a6d3388db431b16a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.535780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eea6d347d99b4765a6d3388db431b16a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.544242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eea6d347d99b4765a6d3388db431b16a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:21.693049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1186/2389] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:21 2022] DELETE /identity/v3/users/eea6d347d99b4765a6d3388db431b16a => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:21.695190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:21.696314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:21.696629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:21.724766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:21.724766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:21.749843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:21.809296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1204/2390] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:21.820193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:21.821683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:21.886831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac29bc3-69e3-4659-affa-d87c8a3a505f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:21.890936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94b304a-6243-41bc-a51f-e09274ed2d05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:21.891098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b304a-6243-41bc-a51f-e09274ed2d05 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:21.891346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b304a-6243-41bc-a51f-e09274ed2d05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:21.891803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94b304a-6243-41bc-a51f-e09274ed2d05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:21.902229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b304a-6243-41bc-a51f-e09274ed2d05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:21.903658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94b304a-6243-41bc-a51f-e09274ed2d05 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:21.914414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1187/2391] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:22.956296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:22.960457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.013389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c32a2-0e90-47d7-852f-2ae91bd5defa None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.018903 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.019826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.020390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.020830 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] PATH_INFO: `/v3/projects/0ecb3ee1dceb40f68765b515234f14b6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.022048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ecb3ee1dceb40f68765b515234f14b6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.024120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.249285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ecb3ee1dceb40f68765b515234f14b6', 'request_id': 'req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.258380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ecb3ee1dceb40f68765b515234f14b6', 'request_id': 'req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.389840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:23.491748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ecb3ee1dceb40f68765b515234f14b6', 'request_id': 'req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.503647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] The token cache is being invalidate because project 0ecb3ee1dceb40f68765b515234f14b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:23.504065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.504761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-9bc54a3f-0b31-4e62-89fe-c1f76ddd1763 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:23.506342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1205/2392] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:22 2022] DELETE /identity/v3/projects/0ecb3ee1dceb40f68765b515234f14b6 => generated 0 bytes in 552 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:23.511435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.511920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.512072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.532024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:23.533249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.596891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1188/2393] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:23.605867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.606224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.606573 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.627696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:23.629440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:23.637331 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1206/2394] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:23.641607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.652401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f1851-6f0f-49b0-883b-1e887c843160 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.657814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.658430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.658711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.659004 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.659713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:23.660115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.661887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.674280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1189/2395] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:23.679589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.690099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb488916-942f-43bc-a8ef-c3018cb84e77 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.695596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.696403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.696684 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.696972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.697643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.699568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:23.850249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1207/2396] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/projects => generated 391 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:23.851812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:23.864776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cc85e-7115-49af-a132-cbd2dbd92aa9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:23.932287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:23.933256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:23.933256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:23.933430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:23.934057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:23.935949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.128063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1190/2397] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:23 2022] POST /identity/v3/users => generated 296 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:24.133124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.143681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85390bf-21bd-4702-90c5-f75f6b6f02df admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.148720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.149397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.150128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.150128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.150617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.152248 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.165884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1208/2398] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:24.167179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.178267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d93345-3f33-4f12-9d54-3da38206fcc8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.183180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75cc0ab-c67b-4a87-a077-b98a63ad03c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.183796 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cc0ab-c67b-4a87-a077-b98a63ad03c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.184080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cc0ab-c67b-4a87-a077-b98a63ad03c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.184355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75cc0ab-c67b-4a87-a077-b98a63ad03c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.185028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75cc0ab-c67b-4a87-a077-b98a63ad03c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.186631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75cc0ab-c67b-4a87-a077-b98a63ad03c3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.198722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1191/2399] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:24.208747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.219169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b9b18-3c9a-4970-a1f0-4ddda0f115c4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.224442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.225135 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.225423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.226052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] PATH_INFO: `/v3/projects/96ebd7bd267848ab8f6d5624cb9656fb/users/094d401dafd64c61bbe9560ac0cd0d91/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.241891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96ebd7bd267848ab8f6d5624cb9656fb, user_id=094d401dafd64c61bbe9560ac0cd0d91, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.244932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.447095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1209/2400] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:24 2022] PUT /identity/v3/projects/96ebd7bd267848ab8f6d5624cb9656fb/users/094d401dafd64c61bbe9560ac0cd0d91/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:24.458446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.458737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.459014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.488761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] MFA Rules not processed for user `094d401dafd64c61bbe9560ac0cd0d91`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:24.490022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.557473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1192/2401] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:24 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:24.568804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:24.570558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:24.628389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f462b7fc-b34e-4fc0-b5f5-414a76201ae4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:24.634167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:24.635263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:24.635263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:24.635506 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:24.644431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:24.645950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:24.654964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1210/2402] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:24 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:24.720917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:25.749340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:26.384334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:26.618267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:27.853686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:31.053308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:31.823528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:32.270473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:33.189306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:35.109351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:36.520306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:36.752765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:37.089516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:37.432380 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:37.627166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:37.783291 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:37.817715 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:38.001005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:43.711824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:43.713407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:43.726049 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:43.782507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dae27db-ea88-4236-97f5-5fb58a587988 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:43.787191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:43.788075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:43.788390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:43.788814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] PATH_INFO: `/v3/users/da157a93c7d84b8d8b84755a47b686bb` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:43.790043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da157a93c7d84b8d8b84755a47b686bb)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:43.792016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:44.054255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da157a93c7d84b8d8b84755a47b686bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.055944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da157a93c7d84b8d8b84755a47b686bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.068331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da157a93c7d84b8d8b84755a47b686bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.069851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da157a93c7d84b8d8b84755a47b686bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.076939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da157a93c7d84b8d8b84755a47b686bb'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.185139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1193/2403] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:43 2022] DELETE /identity/v3/users/da157a93c7d84b8d8b84755a47b686bb => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:44.470379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:44.482231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc35ff0f-ba0d-4733-babe-f69c144681a5 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:44.543547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:44.544190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:44.544471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:44.544746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] PATH_INFO: `/v3/projects/fa04fb474091433bb6e4f1cd939934fa` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:44.545965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa04fb474091433bb6e4f1cd939934fa)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:44.547918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:44.669654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa04fb474091433bb6e4f1cd939934fa', 'request_id': 'req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.675312 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa04fb474091433bb6e4f1cd939934fa', 'request_id': 'req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.754449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa04fb474091433bb6e4f1cd939934fa', 'request_id': 'req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.756745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:44.766532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] The token cache is being invalidate because project fa04fb474091433bb6e4f1cd939934fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:44.766827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.767504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:44.769003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1211/2404] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:44 2022] DELETE /identity/v3/projects/fa04fb474091433bb6e4f1cd939934fa => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:44.773248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:44.774767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:44.788377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ec106d-31bb-49c4-a9a2-b2ab402da807 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:44.847339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:44.847897 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:44.848177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:44.848452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] PATH_INFO: `/v3/users/c7455dbd9f98408ea6d923ba32c916b4` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:44.849626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c7455dbd9f98408ea6d923ba32c916b4)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:44.852291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:45.115347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:45.129556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c7455dbd9f98408ea6d923ba32c916b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.130223 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c7455dbd9f98408ea6d923ba32c916b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.143355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c7455dbd9f98408ea6d923ba32c916b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.144079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c7455dbd9f98408ea6d923ba32c916b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.157273 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-8fa3124e-dd77-491b-a2df-f1e00bd9be98 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c7455dbd9f98408ea6d923ba32c916b4'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.308675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1194/2405] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:44 2022] DELETE /identity/v3/users/c7455dbd9f98408ea6d923ba32c916b4 => generated 0 bytes in 536 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:45.610614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:45.612092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:45.625821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb33349-d6ca-4b20-a5b2-7f08d60fb6ce admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:45.695229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:45.695865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:45.696145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:45.696413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] PATH_INFO: `/v3/projects/06da8cc624c54a3b99cc8880bcda7a31` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:45.697564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06da8cc624c54a3b99cc8880bcda7a31)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:45.699448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:45.857714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06da8cc624c54a3b99cc8880bcda7a31', 'request_id': 'req-544a7ee6-431f-4442-a3a8-a72423232360'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.863134 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06da8cc624c54a3b99cc8880bcda7a31', 'request_id': 'req-544a7ee6-431f-4442-a3a8-a72423232360'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:45.997107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06da8cc624c54a3b99cc8880bcda7a31', 'request_id': 'req-544a7ee6-431f-4442-a3a8-a72423232360'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:46.008966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] The token cache is being invalidate because project 06da8cc624c54a3b99cc8880bcda7a31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:46.009287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:46.009958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-544a7ee6-431f-4442-a3a8-a72423232360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:46.011464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1212/2406] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:45 2022] DELETE /identity/v3/projects/06da8cc624c54a3b99cc8880bcda7a31 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:46.016741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.017303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.017452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.036565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:46.038442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:46.112198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1195/2407] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:46.121253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.121543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.121856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.141852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:46.143067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:46.151247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1213/2408] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:46.155699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.166393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b37efbc-eff4-4a48-a181-37167cb10319 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.171380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.171994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.172280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.172555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.173253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:46.173558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.175244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:46.187915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1196/2409] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:46.192803 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.202478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f7341-4086-457d-9a30-b5fc0c8e5a1b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.207164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.207764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.208045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.208318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.208956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.211138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:46.315373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1214/2410] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:46 2022] POST /identity/v3/projects => generated 375 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:46.316678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.326997 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a580a57-d721-4da8-81e7-be83cef8e58c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.387408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.388012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.388295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.388564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.389234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.391126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:46.531880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1197/2411] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:46 2022] POST /identity/v3/users => generated 288 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:46.535957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.547556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834e8ca-b842-45ad-be9e-85e368cfff59 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.554394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.554394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.554394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.554394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.554972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.556284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:46.567260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1215/2412] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:46.571844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.582560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eae7d0-90b9-4f8a-b319-ec0a60b1b279 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.588281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e32cc8-bb9b-4899-b3f8-f4f424005ee9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.588998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e32cc8-bb9b-4899-b3f8-f4f424005ee9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.589310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e32cc8-bb9b-4899-b3f8-f4f424005ee9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.589580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e32cc8-bb9b-4899-b3f8-f4f424005ee9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.590275 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e32cc8-bb9b-4899-b3f8-f4f424005ee9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.591881 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e32cc8-bb9b-4899-b3f8-f4f424005ee9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:46.604438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1198/2413] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:34:46.621125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.632094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6c577-c197-48d8-ad25-7eaf1b1e7056 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.637988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.638224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.638510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.638852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] PATH_INFO: `/v3/projects/bc5e55eba0bf47bf8ee7e1079b176fe9/users/2505d4d42ec44ba5ab67a604bcd4ef08/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.656861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc5e55eba0bf47bf8ee7e1079b176fe9, user_id=2505d4d42ec44ba5ab67a604bcd4ef08, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.659575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:46.786714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1216/2414] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:46 2022] PUT /identity/v3/projects/bc5e55eba0bf47bf8ee7e1079b176fe9/users/2505d4d42ec44ba5ab67a604bcd4ef08/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:46.798267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.798557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.798827 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.825218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:46.828283 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] MFA Rules not processed for user `2505d4d42ec44ba5ab67a604bcd4ef08`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:46.829555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:46.898853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1199/2415] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:46.903958 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:46.929547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:46.969032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-797ace39-fd83-47e6-9904-4c7a5a84e71f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:46.974554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:46.981413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:46.981413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:46.981413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:46.981413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:46.981413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.079779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1217/2416] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:46 2022] POST /identity/v3/projects => generated 375 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:47.085652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:47.097677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9df62f-776b-4ec6-b4c7-094c39a09fd8 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:47.168728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:47.169371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.169652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.169922 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.170557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:47.172483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.328837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1200/2417] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:47 2022] POST /identity/v3/users => generated 294 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:47.332976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:47.343856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4ae28f-38a5-4fd9-abdf-1a055d09cfdb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:47.349929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:47.349929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.349929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.349929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.350602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:47.352597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.363762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1218/2418] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:47.368680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:47.379599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb70f2-afdb-48b9-b2ac-14db98670218 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:47.385742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:47.396266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.396266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.396266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] PATH_INFO: `/v3/projects/9b6620a1127e4ae1ae3e4edd186998a1/users/89cd492ee8904d03a290a41dca005623/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.403668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6620a1127e4ae1ae3e4edd186998a1, user_id=89cd492ee8904d03a290a41dca005623, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:47.406405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.512758 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1201/2419] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:47 2022] PUT /identity/v3/projects/9b6620a1127e4ae1ae3e4edd186998a1/users/89cd492ee8904d03a290a41dca005623/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:47.522841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:47.533288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30665927-6462-4afc-a6c7-564da41e3013 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:47.594234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a01497e-5b2a-4574-bd21-03cad94b4e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:47.594844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01497e-5b2a-4574-bd21-03cad94b4e31 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.595037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01497e-5b2a-4574-bd21-03cad94b4e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.595336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a01497e-5b2a-4574-bd21-03cad94b4e31 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.595982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01497e-5b2a-4574-bd21-03cad94b4e31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:47.597677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a01497e-5b2a-4574-bd21-03cad94b4e31 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.608746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1219/2420] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:47 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:47.613188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:47.624504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73434e75-2a14-4289-a823-94a24f1447a0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:47.629153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:47.629809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.630083 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.630352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] PATH_INFO: `/v3/projects/9b6620a1127e4ae1ae3e4edd186998a1/users/89cd492ee8904d03a290a41dca005623/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.632885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b6620a1127e4ae1ae3e4edd186998a1, user_id=89cd492ee8904d03a290a41dca005623, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:47.635533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.646305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1202/2421] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:47 2022] PUT /identity/v3/projects/9b6620a1127e4ae1ae3e4edd186998a1/users/89cd492ee8904d03a290a41dca005623/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:47.657773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.658337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.658556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.689319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] MFA Rules not processed for user `89cd492ee8904d03a290a41dca005623`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:47.690533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:47.763882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1220/2422] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:47 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:47.780431 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:47.782212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:47.842213 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaa23a9-ed26-4cff-aea1-bce36986e226 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:47.846257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f914a225-5827-46dd-809f-405fc137920b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:47.846798 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a225-5827-46dd-809f-405fc137920b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:47.847075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a225-5827-46dd-809f-405fc137920b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:47.847347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f914a225-5827-46dd-809f-405fc137920b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:47.855775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:47.903832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914a225-5827-46dd-809f-405fc137920b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:47.905381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f914a225-5827-46dd-809f-405fc137920b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:47.914205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1203/2423] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:52.289964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:52.311303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:52.364722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d6db-e0ff-476c-ba2e-9b8d61ed05b9 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:52.370079 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:52.370710 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:52.370994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:52.371272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] PATH_INFO: `/v3/users/2505d4d42ec44ba5ab67a604bcd4ef08` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:52.372486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=2505d4d42ec44ba5ab67a604bcd4ef08)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:52.374485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:52.608228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2505d4d42ec44ba5ab67a604bcd4ef08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:52.608925 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2505d4d42ec44ba5ab67a604bcd4ef08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:52.609633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2505d4d42ec44ba5ab67a604bcd4ef08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:52.617907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2505d4d42ec44ba5ab67a604bcd4ef08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:52.631788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2505d4d42ec44ba5ab67a604bcd4ef08'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:52.726993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1221/2424] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:52 2022] DELETE /identity/v3/users/2505d4d42ec44ba5ab67a604bcd4ef08 => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:34:52.731701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:52.732188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:52.732281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:52.757863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:52.759109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:52.827850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1204/2425] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:52.837829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:52.839237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:52.907358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a33ed59-fe65-4d91-bb85-c0193fe22fea admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:52.911035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:52.911649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:52.911932 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:52.912285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:52.923479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:52.924918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:52.940118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1222/2426] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:53.504683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:53.504683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:53.517691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e7247b-2875-4061-91cc-0acccc844e3c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:53.522640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:53.523364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:53.523838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:53.524128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] PATH_INFO: `/v3/projects/bc5e55eba0bf47bf8ee7e1079b176fe9` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:53.525461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc5e55eba0bf47bf8ee7e1079b176fe9)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:53.527500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:53.653063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc5e55eba0bf47bf8ee7e1079b176fe9', 'request_id': 'req-80607269-3ce8-46ae-8340-30798e1f8c2f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:53.659319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc5e55eba0bf47bf8ee7e1079b176fe9', 'request_id': 'req-80607269-3ce8-46ae-8340-30798e1f8c2f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:53.668498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc5e55eba0bf47bf8ee7e1079b176fe9', 'request_id': 'req-80607269-3ce8-46ae-8340-30798e1f8c2f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:53.782449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] The token cache is being invalidate because project bc5e55eba0bf47bf8ee7e1079b176fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:53.783091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:53.783634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:53.785201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1205/2427] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:53 2022] DELETE /identity/v3/projects/bc5e55eba0bf47bf8ee7e1079b176fe9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:53.789953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:53.800039 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af09e04-7fe8-4288-9c80-95776482475c service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:53.838907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:53.866829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:53.867816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:53.868862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:53.868862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] PATH_INFO: `/v3/users/89cd492ee8904d03a290a41dca005623` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:53.869706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89cd492ee8904d03a290a41dca005623)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:53.871646 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:54.136536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89cd492ee8904d03a290a41dca005623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:54.137437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89cd492ee8904d03a290a41dca005623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:54.138147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89cd492ee8904d03a290a41dca005623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:54.148136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89cd492ee8904d03a290a41dca005623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:54.160587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-640f1bd9-15d7-4fb4-8848-9b969a8feda1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89cd492ee8904d03a290a41dca005623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:54.312578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1223/2428] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:53 2022] DELETE /identity/v3/users/89cd492ee8904d03a290a41dca005623 => generated 0 bytes in 524 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:54.998832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.000890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:55.017413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80607269-3ce8-46ae-8340-30798e1f8c2f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:55.073078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:55.100475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:55.101739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.101978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.102323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] PATH_INFO: `/v3/projects/9b6620a1127e4ae1ae3e4edd186998a1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.103479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b6620a1127e4ae1ae3e4edd186998a1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:55.105427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:55.285753 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b6620a1127e4ae1ae3e4edd186998a1', 'request_id': 'req-c3e777f5-bfed-4dff-b037-be2c913ccc74'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.292280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b6620a1127e4ae1ae3e4edd186998a1', 'request_id': 'req-c3e777f5-bfed-4dff-b037-be2c913ccc74'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.298784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b6620a1127e4ae1ae3e4edd186998a1', 'request_id': 'req-c3e777f5-bfed-4dff-b037-be2c913ccc74'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.499891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] The token cache is being invalidate because project 9b6620a1127e4ae1ae3e4edd186998a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:55.501743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.501743 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c3e777f5-bfed-4dff-b037-be2c913ccc74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:55.502414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1206/2429] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:54 2022] DELETE /identity/v3/projects/9b6620a1127e4ae1ae3e4edd186998a1 => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:55.507782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.508313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.508313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.528338 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:55.529568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:55.600392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1224/2430] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:55.608949 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.609278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.609550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.629583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:55.630910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:55.639430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1207/2431] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:55.644182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.663046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ec70d-d7ec-475c-901b-5a6ab2c4a5b7 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:55.666651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:55.667301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.667620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.667834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.668483 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:34:55.668950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:55.670937 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:55.688754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1225/2432] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:34:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:34:55.697968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.708371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04a34be-97df-42f9-9ca3-a17c1c07cc05 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:55.714609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:55.715535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.716747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.717842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.719327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:55.721547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:55.864772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1208/2433] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:34:55 2022] POST /identity/v3/projects => generated 365 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:34:55.870205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:55.880416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5757d658-f7a6-4c11-940a-854e77f47304 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:55.925128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:55.939637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:55.940356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:55.940787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:55.941101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:55.941787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:55.944257 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.036276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1226/2434] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:34:55 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:34:56.041265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.052318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67cb9a01-96d5-4da8-9367-48898011371e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.058653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.059337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.059556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.059835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.060605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.062309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.073694 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1209/2435] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:56.078226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.089673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e5edad-7338-4b47-bcfd-04081f8f3c3f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.095160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce6bc00-68b3-46f9-9dad-95537da3495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.095802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6bc00-68b3-46f9-9dad-95537da3495b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.096086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6bc00-68b3-46f9-9dad-95537da3495b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.096360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce6bc00-68b3-46f9-9dad-95537da3495b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.097164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6bc00-68b3-46f9-9dad-95537da3495b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.098907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce6bc00-68b3-46f9-9dad-95537da3495b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.112151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1227/2436] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:34:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:34:56.116833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.127324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d786bc8-069e-4b3b-8ccc-e7969c6a1e7b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.133112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.133793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.134074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.134370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] PATH_INFO: `/v3/projects/b68523c66aa242c5a84a8cddbbbc9f72/users/4fe65ae0d1604a22afec95a3e1c05fe0/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.151658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68523c66aa242c5a84a8cddbbbc9f72, user_id=4fe65ae0d1604a22afec95a3e1c05fe0, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.154474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.251738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1210/2437] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:34:56 2022] PUT /identity/v3/projects/b68523c66aa242c5a84a8cddbbbc9f72/users/4fe65ae0d1604a22afec95a3e1c05fe0/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:56.263075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.263362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.263664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.293384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] MFA Rules not processed for user `4fe65ae0d1604a22afec95a3e1c05fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:56.294597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:56.369225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1228/2438] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:56.377228 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.378982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:56.442869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e417a9-ac2f-43a2-912b-5a4bea745443 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.447358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c2bd01-2a03-42a9-a375-b33770468973 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.447993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2bd01-2a03-42a9-a375-b33770468973 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.448264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2bd01-2a03-42a9-a375-b33770468973 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.448538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c2bd01-2a03-42a9-a375-b33770468973 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.457696 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2bd01-2a03-42a9-a375-b33770468973 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.459119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c2bd01-2a03-42a9-a375-b33770468973 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:56.468751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1211/2439] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:56 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:56.816706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:56.818358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:56.912363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f939b3d6-0b9a-4933-870a-35548d83074c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:56.918459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:56.919202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:56.919547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:56.919893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] PATH_INFO: `/v3/users/094d401dafd64c61bbe9560ac0cd0d91` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:56.921179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] RBAC: Authorizing `identity:delete_user(user_id=094d401dafd64c61bbe9560ac0cd0d91)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:56.923233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.215788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '094d401dafd64c61bbe9560ac0cd0d91'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:57.216446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '094d401dafd64c61bbe9560ac0cd0d91'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:57.217072 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '094d401dafd64c61bbe9560ac0cd0d91'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:57.224198 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '094d401dafd64c61bbe9560ac0cd0d91'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:57.225337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:57.237513 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '094d401dafd64c61bbe9560ac0cd0d91'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:57.379598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1229/2440] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:34:56 2022] DELETE /identity/v3/users/094d401dafd64c61bbe9560ac0cd0d91 => generated 0 bytes in 563 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:34:57.384108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77320190-0836-4c53-b274-4976198e43f0 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.384645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77320190-0836-4c53-b274-4976198e43f0 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.384645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77320190-0836-4c53-b274-4976198e43f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.392152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:34:57.414831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-77320190-0836-4c53-b274-4976198e43f0 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:57.416177 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-77320190-0836-4c53-b274-4976198e43f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:57.490813 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1212/2441] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:57.501560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:57.503136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:57.572078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6a21f9-c91c-45b0-acd7-21abb4bcf226 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:57.577524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:57.578186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:57.578475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:57.578747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:57.592766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:57.594857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:57.610442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1230/2442] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:34:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:34:58.014687 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd9b1-86da-4b30-8cc3-7c273b02d441 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:58.014968 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd9b1-86da-4b30-8cc3-7c273b02d441 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:58.015238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6dfd9b1-86da-4b30-8cc3-7c273b02d441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:58.033997 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e6dfd9b1-86da-4b30-8cc3-7c273b02d441 None None] MFA Rules not processed for user `e037a0abf77f45d7befabbf8d05fcd53`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:34:58.035255 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e6dfd9b1-86da-4b30-8cc3-7c273b02d441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:58.095634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:34:58.100061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1213/2443] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:34:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:34:58.103649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:34:58.140622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05370806-4da7-4259-8375-df175dcabeaa service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:34:58.149982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:34:58.150816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:34:58.151629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:34:58.152110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] PATH_INFO: `/v3/projects/96ebd7bd267848ab8f6d5624cb9656fb` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:34:58.153781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96ebd7bd267848ab8f6d5624cb9656fb)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:34:58.155896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:34:58.324541 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96ebd7bd267848ab8f6d5624cb9656fb', 'request_id': 'req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:58.331438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96ebd7bd267848ab8f6d5624cb9656fb', 'request_id': 'req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:58.427787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96ebd7bd267848ab8f6d5624cb9656fb', 'request_id': 'req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:58.444210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] The token cache is being invalidate because project 96ebd7bd267848ab8f6d5624cb9656fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:34:58.444551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:58.445073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:34:58.447798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1231/2444] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:34:58 2022] DELETE /identity/v3/projects/96ebd7bd267848ab8f6d5624cb9656fb => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:35:08.090775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=51 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:08.438935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=61,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:39.667484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16daa09-0274-4dc5-9229-65f4979a770f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:39.668294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16daa09-0274-4dc5-9229-65f4979a770f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:39.668480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16daa09-0274-4dc5-9229-65f4979a770f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:39.691087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a16daa09-0274-4dc5-9229-65f4979a770f None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:35:39.692395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a16daa09-0274-4dc5-9229-65f4979a770f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:39.753913 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1214/2445] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:35:39 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:35:39.942796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:35:39.944326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:35:40.027442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1e6789-1848-484f-82a1-d98ab0b8d4d6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:35:40.038520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:35:40.038520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:35:40.038520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:35:40.038520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:35:40.043376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:35:40.045590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:35:40.054856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1232/2446] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:35:39 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:35:49.746631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=25 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:35:50.052817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=32 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:02.596995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:02.597624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:02.597624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:02.618247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:02.619540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:02.627864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1215/2447] 10.222.0.73 () {58 vars in 967 bytes} [Tue Aug 23 22:36:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:02.644546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:02.664185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffe619c-d23d-4e7c-b173-39cbaf5d1813 service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:02.673672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:02.674746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:02.675255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:02.675770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:02.688185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:02.689867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:02.699047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1233/2448] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:36:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:36:11.807059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:11.809078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:11.904227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04e996-22f1-45e3-9593-f4048e09542a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:11.909149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:11.909817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:11.910094 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:11.910365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] PATH_INFO: `/v3/users/e037a0abf77f45d7befabbf8d05fcd53` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:11.911641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e037a0abf77f45d7befabbf8d05fcd53)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:11.913624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:12.317649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e037a0abf77f45d7befabbf8d05fcd53'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:12.318370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e037a0abf77f45d7befabbf8d05fcd53'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:12.335659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e037a0abf77f45d7befabbf8d05fcd53'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:12.336576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e037a0abf77f45d7befabbf8d05fcd53'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:12.348621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e037a0abf77f45d7befabbf8d05fcd53'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:12.695328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:12.776441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1216/2449] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:36:11 2022] DELETE /identity/v3/users/e037a0abf77f45d7befabbf8d05fcd53 => generated 0 bytes in 970 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:36:13.212677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:13.233052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98371a90-f5bd-4489-afe0-3726d08efc2a service swift] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:13.312332 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:13.313268 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:13.313747 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:13.314165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] PATH_INFO: `/v3/projects/0fb8b4774d54411099bec473966aced5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:13.315673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0fb8b4774d54411099bec473966aced5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:13.317909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:13.475435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0fb8b4774d54411099bec473966aced5', 'request_id': 'req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:13.483576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0fb8b4774d54411099bec473966aced5', 'request_id': 'req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:13.644727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0fb8b4774d54411099bec473966aced5', 'request_id': 'req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:13.662741 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] The token cache is being invalidate because project 0fb8b4774d54411099bec473966aced5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:36:13.663437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:13.664350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:13.666068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1234/2450] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:36:13 2022] DELETE /identity/v3/projects/0fb8b4774d54411099bec473966aced5 => generated 0 bytes in 454 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:36:14.213084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:14.215332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:14.303993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8631346-3e76-4e54-b73d-de70adf14217 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:14.335911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:14.364410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:14.364978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:14.365308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:14.365586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:14.367369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:14.445530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:14.446984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f5a5b4-354f-472e-9515-b3bc7097591b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:14.460000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1217/2451] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:15.330367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:15.331979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:15.399639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020dbe29-4dc1-4036-a9d0-4eb15ab439a0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:15.404329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:15.404968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:15.405294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:15.405548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] PATH_INFO: `/v3/users/4fe65ae0d1604a22afec95a3e1c05fe0` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:15.406722 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4fe65ae0d1604a22afec95a3e1c05fe0)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:15.408627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:15.612176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:15.628860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4fe65ae0d1604a22afec95a3e1c05fe0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:15.629585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4fe65ae0d1604a22afec95a3e1c05fe0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:15.630275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4fe65ae0d1604a22afec95a3e1c05fe0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:15.640145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4fe65ae0d1604a22afec95a3e1c05fe0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:15.653949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4fe65ae0d1604a22afec95a3e1c05fe0'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:15.746136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1235/2452] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 22:36:15 2022] DELETE /identity/v3/users/4fe65ae0d1604a22afec95a3e1c05fe0 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:36:15.751578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:15.752125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:15.752125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:15.772249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:15.773794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:15.848793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1218/2453] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:15.859901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:15.861698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:15.926311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9ce767-0e62-4b98-804e-2ba9a84afdb1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:15.930183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee955983-be9c-4c09-a8a3-d96d0ca20efe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:15.930798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee955983-be9c-4c09-a8a3-d96d0ca20efe service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:15.931099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee955983-be9c-4c09-a8a3-d96d0ca20efe service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:15.931383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee955983-be9c-4c09-a8a3-d96d0ca20efe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:15.941698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee955983-be9c-4c09-a8a3-d96d0ca20efe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:15.943219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee955983-be9c-4c09-a8a3-d96d0ca20efe service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:15.954649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1236/2454] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:16.423644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:16.426292 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:16.438965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e096ec6-d5bb-4235-a786-fee205868137 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:16.444845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:16.445183 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:16.445365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:16.445639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] PATH_INFO: `/v3/projects/b68523c66aa242c5a84a8cddbbbc9f72` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:16.446878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68523c66aa242c5a84a8cddbbbc9f72)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:16.448869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:16.567085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68523c66aa242c5a84a8cddbbbc9f72', 'request_id': 'req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:16.572838 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68523c66aa242c5a84a8cddbbbc9f72', 'request_id': 'req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:16.580472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68523c66aa242c5a84a8cddbbbc9f72', 'request_id': 'req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:16.583311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:16.681295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] The token cache is being invalidate because project b68523c66aa242c5a84a8cddbbbc9f72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 22:36:16.681530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:16.682435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1b4dfb5e-0ebf-43ac-8b5c-88730da5f226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 22:36:16.684840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1219/2455] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 22:36:16 2022] DELETE /identity/v3/projects/b68523c66aa242c5a84a8cddbbbc9f72 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:36:16.689991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:16.690505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:16.690505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:16.711131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:16.712673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:16.776534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1237/2456] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:16.784717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:16.785432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:16.785432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:16.805691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:16.806909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:16.815658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1220/2457] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:16.823017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:16.834299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc541905-6016-47f8-835b-02bdb42cf13c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:16.839147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:16.840026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:16.840653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:16.840989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:16.841673 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 22:36:16.841977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:16.843653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:16.857842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1238/2458] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 22:36:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 22:36:16.862394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:16.877500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f24ca39-6cf2-4a9f-b4d1-75b4817de6fe None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:16.877500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:16.877869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:16.878116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:16.878388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:16.879041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:16.880939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:17.011943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1221/2459] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:36:16 2022] POST /identity/v3/projects => generated 385 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:36:17.016798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:17.029313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eeb96fd-de6d-4e74-9fc9-01a0007f7cd9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:17.083242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:17.104694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:17.105336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:17.105617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:17.106640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:17.107288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:17.109263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:17.206609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1239/2460] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:36:17 2022] POST /identity/v3/users => generated 293 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:36:17.210998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:17.222263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511e63be-fea0-427b-8356-a897799d2ed1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:17.229818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:17.231500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:17.231500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:17.231737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:17.232309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:17.233945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:17.244839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1222/2461] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:36:17.249945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:17.260151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef96bc9-c472-4cc6-bc26-5123279a90bb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:17.265750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bfb33-39a9-4253-8562-25071a497164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:17.266294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bfb33-39a9-4253-8562-25071a497164 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:17.268028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bfb33-39a9-4253-8562-25071a497164 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:17.268028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bfb33-39a9-4253-8562-25071a497164 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:17.268028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bfb33-39a9-4253-8562-25071a497164 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:17.269682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bfb33-39a9-4253-8562-25071a497164 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:17.286889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1240/2462] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:36:17 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 22:36:17.291385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:17.304021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d9701e-387d-4f44-833c-56cb4fb69374 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:17.308733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:17.309535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:17.309861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:17.310079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] PATH_INFO: `/v3/projects/754172385f004ede86566e52cddf1d47/users/692cc40ea1fe47a9904f1370a8d9ba2a/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:17.326257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] RBAC: Authorizing `identity:create_grant(project_id=754172385f004ede86566e52cddf1d47, user_id=692cc40ea1fe47a9904f1370a8d9ba2a, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:17.329491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:17.414970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1223/2463] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:36:17 2022] PUT /identity/v3/projects/754172385f004ede86566e52cddf1d47/users/692cc40ea1fe47a9904f1370a8d9ba2a/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:36:17.422064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:17.422463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:17.422746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:17.444290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:17.445764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:17.528118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1241/2464] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:36:17.540446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:17.542608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:17.627033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f768c2-38f5-4bfe-9973-f931fc942550 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:17.631453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5481745-21b0-4569-8d7b-47bfd215cd72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:17.632085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5481745-21b0-4569-8d7b-47bfd215cd72 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:17.632388 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5481745-21b0-4569-8d7b-47bfd215cd72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:17.632755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5481745-21b0-4569-8d7b-47bfd215cd72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:17.643871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5481745-21b0-4569-8d7b-47bfd215cd72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:17.645358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5481745-21b0-4569-8d7b-47bfd215cd72 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:17.663209 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1224/2465] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:18.825464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:18.826926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:18.841805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c508738-8004-4c36-aa14-b50d6a625802 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:18.845631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:18.851430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:18.851430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:18.851430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:18.851430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:18.935754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:18.937216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:18.948737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1242/2466] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:18 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:24.258869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:24.259364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:24.259467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:24.297123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] MFA Rules not processed for user `692cc40ea1fe47a9904f1370a8d9ba2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:24.298115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:24.392297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1225/2467] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:24 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:24.396713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:24.408159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6977ed-47f0-4e59-afa2-3af656f2e28c service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:24.413041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:24.413993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:24.413993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:24.414215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:24.414886 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:24.416800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:24.516227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1243/2468] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 22:36:24 2022] POST /identity/v3/projects => generated 383 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 22:36:24.520568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:24.533359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed25fd3-639a-47fd-86b9-0ca5479cb25e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:24.555669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=1,keystone:SELECT=97,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:24.596126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:24.596734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:24.597098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:24.597450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:24.598136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:24.600055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:24.673422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1226/2469] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 22:36:24 2022] POST /identity/v3/users => generated 298 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 22:36:24.677939 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:24.689893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2163dd91-c726-4a8a-b6db-af0f33762b49 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:24.694567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:24.695170 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:24.695452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:24.695719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:24.696366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:24.698010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:24.709544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1244/2470] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:36:24.714861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:24.726330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba1dde-1cf2-43ab-8ef0-a5bc98e2d58b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:24.732746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:24.733465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:24.733764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:24.734126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] PATH_INFO: `/v3/projects/ea56b0c1ecb842a7b3f0144a6fb4cf87/users/bfeb3b40e6354cf482921ef37254eff3/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:24.751669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea56b0c1ecb842a7b3f0144a6fb4cf87, user_id=bfeb3b40e6354cf482921ef37254eff3, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:24.754316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:24.924905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1227/2471] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:36:24 2022] PUT /identity/v3/projects/ea56b0c1ecb842a7b3f0144a6fb4cf87/users/bfeb3b40e6354cf482921ef37254eff3/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 22:36:24.929539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:24.940689 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00dcf233-3267-4578-b089-66fc0937b26d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:24.951104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:25.030668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2dcaed-e20a-4066-9bd8-ab721abd8591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:25.031730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2dcaed-e20a-4066-9bd8-ab721abd8591 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:25.031730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2dcaed-e20a-4066-9bd8-ab721abd8591 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:25.032056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2dcaed-e20a-4066-9bd8-ab721abd8591 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:25.032748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2dcaed-e20a-4066-9bd8-ab721abd8591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:25.034471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2dcaed-e20a-4066-9bd8-ab721abd8591 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:25.048507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1245/2472] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 22:36:24 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 22:36:25.052777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:25.064466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88672d-5016-4891-9e7e-e2d0e36000bf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:25.070158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:25.071409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:25.071689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:25.071966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] PATH_INFO: `/v3/projects/ea56b0c1ecb842a7b3f0144a6fb4cf87/users/bfeb3b40e6354cf482921ef37254eff3/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:25.074159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea56b0c1ecb842a7b3f0144a6fb4cf87, user_id=bfeb3b40e6354cf482921ef37254eff3, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:25.076976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:25.096445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1228/2473] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 22:36:25 2022] PUT /identity/v3/projects/ea56b0c1ecb842a7b3f0144a6fb4cf87/users/bfeb3b40e6354cf482921ef37254eff3/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 22:36:31.672335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eb164-af11-4ebe-a045-eaef89839607 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:31.672335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eb164-af11-4ebe-a045-eaef89839607 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:31.672335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55eb164-af11-4ebe-a045-eaef89839607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:31.718995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c55eb164-af11-4ebe-a045-eaef89839607 None None] MFA Rules not processed for user `bfeb3b40e6354cf482921ef37254eff3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:31.720437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c55eb164-af11-4ebe-a045-eaef89839607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:31.817786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1246/2474] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:31 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:31.827256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:31.829929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:31.898322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede4d3e8-c3bd-44af-9e6e-f0a6f008d84a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:31.902310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:31.902990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:31.903325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:31.903757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:31.984957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:31.988418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:31.997960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1229/2475] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:32.273279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143962-8146-4e22-8703-02881f24b8eb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:32.273562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143962-8146-4e22-8703-02881f24b8eb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:32.273839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee143962-8146-4e22-8703-02881f24b8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:32.295486 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ee143962-8146-4e22-8703-02881f24b8eb None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:32.296732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ee143962-8146-4e22-8703-02881f24b8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:32.377836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1247/2476] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:36:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:32.392505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:32.392505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:32.428128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:32.472873 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b035c8b-fb2a-4429-a473-e4ca87524047 service glance] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:32.478788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:32.479424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:32.479703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:32.480976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:32.490323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:32.491952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:32.501763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1230/2477] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:36:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:36:35.860128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:35.860656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:35.860656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:35.882356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] MFA Rules not processed for user `adc12894a7c94ae1b06340353fd99654`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:35.883618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:35.927058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:35.961707 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1248/2478] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:35 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:36.259666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:36.348410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32929c95-7bbb-4a06-9346-7a7e0930c91d service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:36.352345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:36.353146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:36.353498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:36.353846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:36.355993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:36.368365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:36.369962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:36.385224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1231/2479] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:36.946508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:36.949733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:37.048933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ba9f25-66de-47ee-9311-b8cc7686e042 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:37.055429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:37.056140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:37.056487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:37.056898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:37.059168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:37.072398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:37.074512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:37.088975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1249/2480] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:36 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:41.436312 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:41.447995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f973eb50-7a4d-4eb1-9401-6f816d17f11b service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:41.453793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:41.454506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:41.454962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:41.455395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:41.457350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:41.469745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:41.477656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e144ef-f8d5-4f3e-b9b1-cb704cd81b28 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:41.500070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1232/2481] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:42.823202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:42.824966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:42.945695 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b0a23-6b61-4f84-b78e-23cabe911f0b service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:42.966931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:42.967459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:42.967737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:42.968019 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:42.970110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:42.983399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:42.985215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c5942e-5bf0-4f28-821c-b50bcc4377d5 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:42.997353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1250/2482] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:36:44.792561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:44.793369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:44.793369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:44.814485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] MFA Rules not processed for user `692cc40ea1fe47a9904f1370a8d9ba2a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:44.815739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:44.823225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1233/2483] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:36:44 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:36:49.666251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:49.667792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:49.667792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:49.693190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:36:49.694487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:49.703467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1251/2484] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:36:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:36:49.718337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:49.734397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d198b7b7-a60a-4c0f-baeb-ac997110fa6e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:49.740912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:49.742019 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:49.742295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:49.742729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:49.743865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:49.754256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:49.756555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:49.764988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65bdacf-d069-4aaa-b1f2-a2af5d8ffb7d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:49.771383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1234/2485] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:36:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:36:49.773850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:49.774659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:49.775219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:49.775780 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:49.779251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:49.848880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:36:49.861920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:49.863570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:49.873672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1252/2486] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:36:50.719979 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:36:50.727110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:50.763712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aa089f-db2a-48fb-b0d7-e902a5e39653 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:36:50.774163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:36:50.776105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:36:50.776925 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:36:50.778570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:36:50.782402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:36:50.810989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:36:50.815078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:36:50.839317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1235/2487] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:36:50 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:37:00.198393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:37:00.698192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:37:00.710267 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c80f362-ec61-44f8-9769-1505b8c8c5c7 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:37:00.717224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:37:00.717986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:37:00.718265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:37:00.718538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:37:00.720349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:37:00.787823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:37:00.789535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7420c994-3eec-40f8-a940-b56a4f35269e service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:37:00.800519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1253/2488] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:37:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:37:00.829154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=66 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:37:10.795770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=28 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:38:34.549418 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:34.560188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40102a6f-cb3f-40a4-91ae-d2cd73ae3ae4 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:34.564725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:34.565398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:34.566247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:34.566519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:34.576833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:34.578309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:34.593214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1236/2489] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:34 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:35.425959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca38d54-c056-43d3-86c7-264e00741a28 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:35.426574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca38d54-c056-43d3-86c7-264e00741a28 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:35.426574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca38d54-c056-43d3-86c7-264e00741a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:35.451935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-cca38d54-c056-43d3-86c7-264e00741a28 None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:38:35.453241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-cca38d54-c056-43d3-86c7-264e00741a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:35.463067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1254/2490] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:38:35 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:38:35.519482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:35.547847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ef1d53-7df3-4095-9239-b5abe7851f09 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:35.569850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:35.571184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:35.571677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:35.572143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:35.586396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:35.679066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:35.679066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1237/2491] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:38:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:38:38.306061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:38.306061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:38.306061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:38.344277 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] MFA Rules not processed for user `ebbf49534a8f41109f51637307270016`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:38:38.346462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:38.356928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1255/2492] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:38:38 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:38:38.433564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:38.452615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e070aef-146f-4c07-829a-f2785534a994 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:38.466397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:38.467520 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:38.469257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:38.469690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:38.475905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:38.608272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:38.610121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba10f6dd-d1ef-4663-b95b-0bb9c215f748 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:38.623120 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1238/2493] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:43.347422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:43.366308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6dd417-a5f2-4fd5-90f2-68c9d277c507 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:43.374123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:43.374919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:43.375834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:43.376238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:43.378890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:43.401147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:43.401147 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:43.425619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1256/2494] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:43 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:45.288662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b996e94-5a1f-4499-af20-6dc0f84e7f97 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:45.289609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b996e94-5a1f-4499-af20-6dc0f84e7f97 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:45.290037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b996e94-5a1f-4499-af20-6dc0f84e7f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:45.318622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-6b996e94-5a1f-4499-af20-6dc0f84e7f97 None None] MFA Rules not processed for user `2ca578d6a1374212a4f1476335c641b3`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:38:45.321387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6b996e94-5a1f-4499-af20-6dc0f84e7f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:45.341279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1239/2495] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:38:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:38:46.195986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:46.212694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ad0a88-44fd-44df-8cd4-73630efbbf44 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:46.224061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:46.225061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:46.225577 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:46.226058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:46.230453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:46.243551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:46.245464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e68eba-eb5c-485e-be6f-b35059a676ab service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:46.256280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1257/2496] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:46 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:55.309700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=43 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:38:56.250817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=18 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:38:57.365341 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd0b1-617b-4847-af55-80619a332f92 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.366533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd0b1-617b-4847-af55-80619a332f92 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.366809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1fd0b1-617b-4847-af55-80619a332f92 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.368351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1240/2497] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:38:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:38:57.375405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd4745-3c11-49de-8f1e-53b19f873092 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.375890 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd4745-3c11-49de-8f1e-53b19f873092 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.376273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedd4745-3c11-49de-8f1e-53b19f873092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.403629 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-aedd4745-3c11-49de-8f1e-53b19f873092 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:38:57.404880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aedd4745-3c11-49de-8f1e-53b19f873092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:57.414769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1258/2498] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:38:57.426158 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fd0b1-617b-4847-af55-80619a332f92 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:57.437400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1fd0b1-617b-4847-af55-80619a332f92 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:57.440946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfef648-f0ff-47c9-9717-8743cb299624 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:57.441627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef648-f0ff-47c9-9717-8743cb299624 service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.441904 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef648-f0ff-47c9-9717-8743cb299624 service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.442173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfef648-f0ff-47c9-9717-8743cb299624 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.452437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef648-f0ff-47c9-9717-8743cb299624 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:57.453894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfef648-f0ff-47c9-9717-8743cb299624 service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:57.465879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1241/2499] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:57 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:57.741908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e89987-6399-4ed4-899d-69e632d06a8e None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.742199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e89987-6399-4ed4-899d-69e632d06a8e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.742478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e89987-6399-4ed4-899d-69e632d06a8e None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.743990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1259/2500] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:38:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:38:57.749793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c2e8d-865c-447b-984c-f8325c141a84 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.749995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c2e8d-865c-447b-984c-f8325c141a84 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.750278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017c2e8d-865c-447b-984c-f8325c141a84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.771731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-017c2e8d-865c-447b-984c-f8325c141a84 None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:38:57.772947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-017c2e8d-865c-447b-984c-f8325c141a84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:57.781221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1242/2501] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:38:57.798991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e89987-6399-4ed4-899d-69e632d06a8e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:57.810482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e89987-6399-4ed4-899d-69e632d06a8e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:57.814208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82fb81e-071e-40f3-860f-7d65ffea67d9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:57.814750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb81e-071e-40f3-860f-7d65ffea67d9 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.815035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb81e-071e-40f3-860f-7d65ffea67d9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.815302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82fb81e-071e-40f3-860f-7d65ffea67d9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.823572 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fb81e-071e-40f3-860f-7d65ffea67d9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:57.824992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82fb81e-071e-40f3-860f-7d65ffea67d9 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:57.834075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1260/2502] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:57 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:38:57.956749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bdbcb-8f0e-4dbc-ab52-a4ca1bee05d3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.957217 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bdbcb-8f0e-4dbc-ab52-a4ca1bee05d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.957755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0bdbcb-8f0e-4dbc-ab52-a4ca1bee05d3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.959139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1243/2503] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 22:38:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:38:57.965064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:57.965349 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:57.965616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:57.986680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:38:57.987909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:38:57.999237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1261/2504] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 22:38:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 22:38:58.011466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bdbcb-8f0e-4dbc-ab52-a4ca1bee05d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:38:58.023108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0bdbcb-8f0e-4dbc-ab52-a4ca1bee05d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:38:58.026781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:38:58.027414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:38:58.027691 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:38:58.027961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:38:58.038789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:38:58.040253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:38:58.051867 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1244/2505] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:38:58 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:39:07.981580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=12 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:39:08.046415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=15 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:39:15.691896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:39:15.693471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:15.781531 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2d20b8-b09f-4f75-951b-d141461deef0 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:39:15.785987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:39:15.785987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:39:15.786860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:39:15.786975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:39:15.788781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:39:15.816209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:39:15.817665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:39:15.826778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1262/2506] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:39:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:39:25.822503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=45 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:40:10.992570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:40:11.045659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b09ff9-a7c8-4eef-9c9d-ffe86c6e46a3 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:40:11.051454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:40:11.052056 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:40:11.052338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:40:11.052614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:40:11.054409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:40:11.078317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:40:11.080152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:40:11.090959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1245/2507] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:40:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:40:21.086072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=29 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:41:15.735586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:41:15.756360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3a5ecf-eea8-4087-8863-179ce6d9a738 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:41:15.760103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:41:15.760735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:41:15.761059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:41:15.761364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:41:15.763116 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:41:15.798027 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:41:15.801337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:41:15.821478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1263/2508] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:41:15 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:41:19.375242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:41:19.384811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe271ee-13c0-410c-9a27-dc4ed7802caa service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:41:19.388395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:41:19.388985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:41:19.389291 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:41:19.389564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:41:19.399685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:41:19.401134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:41:19.412422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1246/2509] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:41:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:41:25.818900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=22 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:41:29.407093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:41:31.726538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:41:31.737551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd9feb3-6e6e-4a5e-8943-86476e0bbd01 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:41:31.743954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:41:31.744546 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:41:31.744825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:41:31.745130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:41:31.752934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:41:31.754584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:41:31.764224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1264/2510] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:41:31 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:41:41.760167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:41:49.292080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:41:49.304948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4ee9c2-9bf7-43d0-bec7-60a4464b8143 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:41:49.311901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:41:49.314358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:41:49.315175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:41:49.315741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:41:49.329927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:41:49.332857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:41:49.364955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1247/2511] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:41:49 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:41:59.361907 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=13 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:42:00.905128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:42:00.929537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06860bf-53fb-42d0-ba6f-86dc8af1ac6d service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:42:00.934680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:42:00.935339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:42:00.935703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:42:00.935884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:42:00.946668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:42:00.949677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:42:00.980539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1265/2512] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:42:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:42:08.537290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:42:08.549934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0859f3f-17b7-49a2-bb56-220f4b7dc00b service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:42:08.553993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:42:08.554607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:42:08.554924 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:42:08.555192 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:42:08.563692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:42:08.565214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:42:08.574029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1248/2513] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:42:08 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:42:10.977274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=17 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:42:18.569802 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:43:38.567885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:43:38.582767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5a6baa-02c6-438d-af38-202022956cc2 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:43:38.588772 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411d82c1-77dd-44bc-8bac-9aa25b2704b1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:43:38.589387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d82c1-77dd-44bc-8bac-9aa25b2704b1 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:43:38.589677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d82c1-77dd-44bc-8bac-9aa25b2704b1 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:43:38.590021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411d82c1-77dd-44bc-8bac-9aa25b2704b1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:43:38.600181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d82c1-77dd-44bc-8bac-9aa25b2704b1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:43:38.601813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411d82c1-77dd-44bc-8bac-9aa25b2704b1 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:43:38.639015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1266/2514] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:43:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:43:48.635677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=13 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:44:15.731871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:44:15.733313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:44:15.757748 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31567951-5834-4597-8863-0bf4b864a8e1 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:44:15.761739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:44:15.762344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:44:15.762618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:44:15.762888 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:44:15.772222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:44:15.777575 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:44:15.801857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1249/2515] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:44:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:44:25.799143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=18 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:44:57.873466 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:44:57.874051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:44:57.874051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:44:57.935389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:44:57.936639 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:44:57.954894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1267/2516] 10.222.0.73 () {58 vars in 967 bytes} [Tue Aug 23 22:44:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:44:58.147225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:44:58.169645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff91cce-db3c-4b7d-851c-d5eb98e950ad service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:44:58.176757 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5792d7-b76d-4b35-a496-7dfe071fbc3c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:44:58.177414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5792d7-b76d-4b35-a496-7dfe071fbc3c service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:44:58.177700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5792d7-b76d-4b35-a496-7dfe071fbc3c service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:44:58.178005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5792d7-b76d-4b35-a496-7dfe071fbc3c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:44:58.191774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5792d7-b76d-4b35-a496-7dfe071fbc3c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:44:58.193428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5792d7-b76d-4b35-a496-7dfe071fbc3c service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:44:58.201635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1250/2517] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:44:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:45:01.096511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:01.112394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dc60e7-f5d9-4f7f-8c26-6c0d4d68091c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:01.118754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-277c14e9-2834-4568-a072-8ec18f2d05c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:01.119644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c14e9-2834-4568-a072-8ec18f2d05c5 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:01.119983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c14e9-2834-4568-a072-8ec18f2d05c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:01.120323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-277c14e9-2834-4568-a072-8ec18f2d05c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:01.130652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c14e9-2834-4568-a072-8ec18f2d05c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:01.132298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-277c14e9-2834-4568-a072-8ec18f2d05c5 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:01.143507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1268/2518] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:01.329146 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436aac2-98b7-43e2-bddf-1921c9f407f3 None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:01.329795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436aac2-98b7-43e2-bddf-1921c9f407f3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:01.329795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4436aac2-98b7-43e2-bddf-1921c9f407f3 None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:01.331729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1251/2519] 10.222.0.73 () {54 vars in 912 bytes} [Tue Aug 23 22:45:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 22:45:01.338328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caee43-bb1c-4f8b-bee1-4dbeafdb9701 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:01.338815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caee43-bb1c-4f8b-bee1-4dbeafdb9701 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:01.339270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10caee43-bb1c-4f8b-bee1-4dbeafdb9701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:01.408324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-10caee43-bb1c-4f8b-bee1-4dbeafdb9701 None None] MFA Rules not processed for user `f077a58c9406414085dba8d67c69e364`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:45:01.409699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-10caee43-bb1c-4f8b-bee1-4dbeafdb9701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:45:01.419228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1269/2520] 10.222.0.73 () {58 vars in 1009 bytes} [Tue Aug 23 22:45:01 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:45:01.445136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436aac2-98b7-43e2-bddf-1921c9f407f3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:01.459739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4436aac2-98b7-43e2-bddf-1921c9f407f3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:01.464669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:01.465652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:01.465927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:01.466195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:01.475535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:01.477413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:01.487610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1252/2521] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:05.341494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:05.343105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:05.343105 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:05.384969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] MFA Rules not processed for user `3eda1fca08b24727929b63318fd6f13e`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:45:05.386200 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:45:05.393773 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1270/2522] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 22:45:05 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:45:05.711994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:05.722648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552e59cb-f6f0-4e9d-84bb-2c1304b7d56f service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:05.726009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:05.726855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:05.727038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:05.727306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:05.735517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:05.736962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:05.744885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1253/2523] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:05 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:06.246898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:06.248397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:45:06.270144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a30d3-3af7-4470-bb40-9b6f866ece13 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:06.276210 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:06.276837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:06.277165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:06.277445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:06.290015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:06.291681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:06.302040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1271/2524] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:06 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:14.216434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:14.236045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b9e645-f9a1-4ba9-a78b-cb256e390659 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:14.242494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:14.243096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:14.243370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:14.243697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:14.262091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:14.263692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:14.276334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1254/2525] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:14 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:16.297510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=58 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:45:16.806142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:16.827330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3060a7e8-90b6-463a-b316-55eed1d4a664 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:16.827330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:16.827330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:16.827330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:16.827832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:16.837310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:16.839961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:16.852802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1272/2526] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:21.816132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:45:21.829901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a62b43-6912-4f4a-be28-0de555ada8ef service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:45:21.835678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:45:21.836379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:45:21.836722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:45:21.837122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:45:21.847321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:45:21.849132 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:45:21.861441 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1255/2527] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:45:21 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:45:26.848605 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:45:31.855816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=34 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:46:19.497618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:46:19.506859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125d5e09-888c-45d4-9225-621283ed18c2 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:46:19.510382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:46:19.510969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:46:19.511242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:46:19.511509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:46:19.513414 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:46:19.526631 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:46:19.528041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:46:19.538821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1273/2528] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:46:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:46:29.533964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:46:32.405195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:46:32.414500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f434de-db1d-4ef3-85fd-c655a9aa8541 service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:46:32.418749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:46:32.419346 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:46:32.419626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:46:32.419898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:46:32.421774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:46:32.505890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:46:32.507369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:46:32.524825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1256/2529] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:46:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:46:42.522263 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=41 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:46:50.006262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:46:50.018891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7703d6-2333-4ee2-9856-5af2e0b6bfba service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:46:50.024127 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:46:50.024997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:46:50.025458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:46:50.026470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:46:50.029284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:46:50.294563 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:46:50.296049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:46:50.305788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1274/2530] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:46:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:47:00.301263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=33 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:47:01.133643 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:47:01.143736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5499c8f-cee7-46a3-8f9b-eacd216a5ff8 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:47:01.147558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:47:01.149531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:47:01.149531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:47:01.149531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:47:01.150787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:47:01.230097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:47:01.231540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:47:01.241075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1257/2531] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:47:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:47:10.537497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:47:10.539211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:47:10.610047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5763ee-04f7-46f2-90a9-e907cdb18a3c service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:47:10.614602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:47:10.615219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:47:10.615520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:47:10.615769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:47:10.617655 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:47:10.629499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:47:10.631040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:47:10.644894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1275/2532] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:47:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:47:11.236678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=33 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:47:20.636529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=31 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:48:38.543340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:48:38.552395 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9256e18e-7627-4bfe-a18f-0f8f5d7b9bb7 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:48:38.556204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:48:38.557702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:48:38.557702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:48:38.557702 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:48:38.569224 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:48:38.570763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:48:38.580765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1258/2533] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:48:38 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:48:48.575769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:49:16.575036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:49:16.576825 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:49:16.662377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b24460-08d2-404a-a141-46b26b7eb091 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:49:16.666048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:49:16.666662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:49:16.666950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:49:16.667224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:49:16.669845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:49:16.743716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:49:16.745169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:49:16.754452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1276/2534] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:49:16 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:49:26.750241 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=61 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:49:35.236231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:49:35.246551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1a17b6-7d1d-4cb9-85e1-296ec6ac197a service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:49:35.250116 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:49:35.250708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:49:35.250983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:49:35.251249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:49:35.318009 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:49:35.319348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:49:35.330875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1259/2535] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:49:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:49:45.325765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=34 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:50:19.863862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:50:19.981366 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fda0ff-17a8-4d3c-b989-ace6febdc410 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:50:19.987070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:50:19.987674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:50:19.987948 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:50:19.988225 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:50:19.990104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:50:20.014705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:50:20.016274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:50:20.025894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1277/2536] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:50:19 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:50:30.021767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=55 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:51:20.261121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:51:20.263048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:51:20.303818 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3523d2ac-1598-4c2f-9417-74b466a62a06 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:51:20.310265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:51:20.310887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:51:20.311170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:51:20.311444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:51:20.329789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:51:20.331317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:51:20.354022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1260/2537] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:51:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:51:30.351276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=22 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:51:32.557933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:51:32.567290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dd6fa4-30a6-4e75-b8d2-9fe1ff339527 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:51:32.570845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:51:32.571431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:51:32.571709 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:51:32.571978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:51:32.581771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:51:32.583218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:51:32.593307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1278/2538] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:51:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:51:42.588434 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:51:50.913586 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:51:50.924100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0b5b11-a54f-4a6a-b7b8-c7d9dc5dc123 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:51:50.927801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:51:50.928410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:51:50.928682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:51:50.928951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:51:50.939127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:51:50.940579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:51:50.969101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1261/2539] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:51:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:52:00.966231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=13 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:52:01.633805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:52:01.652148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ff6e9a-e3a8-48cb-959e-33e9085d74e6 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:52:01.655745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:52:01.657064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:52:01.657530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:52:01.657807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:52:01.666276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:52:01.667686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:52:01.695085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1279/2540] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:52:01 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:52:05.506233 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:52:05.508067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:52:05.537561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e37ace-e3ca-43bc-94a8-3cfb970dffb3 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:52:05.543911 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:52:05.544497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:52:05.544777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:52:05.545079 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:52:05.548198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:52:05.562154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:52:05.564808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:52:05.578025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1262/2541] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:52:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:52:11.692463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=17 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:52:15.537591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:52:15.547359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d239b73d-799b-427d-b1ab-770c4251711e service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:52:15.551808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:52:15.552403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:52:15.552679 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:52:15.552945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:52:15.560670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:52:15.562108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:52:15.570288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1280/2542] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:52:15 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 33 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:52:15.572762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=11 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:52:25.566668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:54:06.013150 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:54:06.023424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac87590-35d0-47ee-b6af-3d58aaf6339c service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:54:06.027063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:54:06.027659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:54:06.027940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:54:06.028266 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:54:06.030328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:54:06.102327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:54:06.103776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:54:06.123548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1263/2543] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:54:06 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:54:15.896692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:54:15.909452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019cdd42-0386-46c2-991f-590aa27a8ab7 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:54:15.914815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:54:15.915398 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:54:15.915675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:54:15.915942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:54:15.924080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:54:15.925670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:54:15.954160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1281/2544] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:54:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:54:16.120833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=36 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:54:25.950945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=13 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:54:35.052323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:54:35.063449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c8da98-f91a-414a-be78-3afbdaeaf0af service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:54:35.067481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:54:35.068164 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:54:35.068515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:54:35.068797 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:54:35.070731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:54:35.086387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:54:35.087833 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:54:35.101286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1264/2545] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:54:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:54:45.095265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:55:26.834357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:55:26.835979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:55:26.851474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef9eef4-8bc7-4390-9eae-8d05bb7274dc service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:55:26.856918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:55:26.857579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:55:26.857913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:55:26.858876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:55:26.870598 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:55:26.872224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:55:26.883045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1282/2546] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:55:26 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 22:55:36.878306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:56:20.590133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:56:20.599926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6302dbb9-4c0f-480c-9a1c-14a1dc6e8b72 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:56:20.603739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:56:20.604332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:56:20.604612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:56:20.604886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:56:20.606772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:56:20.621071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:56:20.623880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:56:20.635278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1265/2547] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:56:20 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:56:30.630289 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:56:32.986836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:56:32.988545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:56:33.004875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b7af5-8066-405a-a182-b02ed61f27ca service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:56:33.010403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:56:33.011066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:56:33.011342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:56:33.011614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:56:33.013498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:56:33.102014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:56:33.103649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:56:33.125202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1283/2548] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:56:32 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:56:43.121847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=41 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:56:50.684042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:56:50.695846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781da2da-2297-4e59-9ca6-33ce70942ce7 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:56:50.701369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:56:50.701975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:56:50.702835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:56:50.703129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:56:50.704980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:56:50.776707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:56:50.778290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:56:50.788071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1266/2549] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:56:50 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:57:00.783383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=33 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:57:02.143106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:57:02.147083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:57:02.169534 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9743c05b-7b6c-4176-8c08-28e7f41ba3f2 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:57:02.173831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:57:02.174435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:57:02.174720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:57:02.174990 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:57:02.176818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:57:02.248300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:57:02.250374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5178d224-052c-4df4-966a-6a379da12ece service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:57:02.259797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1284/2550] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:57:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:57:12.255800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=38 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:57:18.542895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:57:18.544610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:57:18.617812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e31b947-1fb8-4b94-a958-474c13d6a23e service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:57:18.621558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:57:18.622169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:57:18.622448 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:57:18.623327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:57:18.625648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:57:18.635387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:57:18.636817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:57:18.646248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1267/2551] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:57:18 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:57:28.642584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=31 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:59:02.872807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:59:02.873389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:59:02.873520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:59:02.933415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] MFA Rules not processed for user `da3e06d942754501b4c903e34e6bbc9d`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 22:59:02.934519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:59:03.000301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1285/2552] 10.222.0.73 () {58 vars in 967 bytes} [Tue Aug 23 22:59:02 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 22:59:03.175804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:59:03.177478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:59:03.277035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cb5e28-3b95-4675-aef1-134cc0a1456c service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:59:03.282457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a54377901631412abe887223a71d3623', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:59:03.283077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:59:03.283358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:59:03.283635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:59:03.292478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:59:03.294109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:59:03.302779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1268/2553] 10.222.0.73 () {60 vars in 1384 bytes} [Tue Aug 23 22:59:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 23 22:59:04.720456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 22:59:04.731580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfce003b-3ea6-45df-b530-75b012f95804 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 22:59:04.735781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 22:59:04.737004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 22:59:04.737793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 22:59:04.738081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 22:59:04.741350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 22:59:04.836490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 22:59:04.838480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 22:59:04.858501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1286/2554] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 22:59:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 22:59:13.299226 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=41 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 22:59:14.855759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:07.255814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:07.257477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:07.337545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4758f6-1d0c-483f-8a71-3463680b7f56 service swift] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:07.341097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:07.341700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:07.341975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:07.342242 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:07.344089 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:07.415117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:07.416528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:07.426211 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1269/2555] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:00:07 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:00:07.471197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:07.480857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa485f9-ec4e-435f-9045-e7b30efb86dc service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:07.485362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:07.485967 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:07.486250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:07.486524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:07.496501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:07.498004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:07.507304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1287/2556] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:00:07 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:00:07.998871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:07.998871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab74245-230b-4b03-86b7-a43344fffb3b service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:07.998871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:07.998871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:08.000927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:08.000927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:08.000927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:08.000927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:08.000927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:08.000927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1270/2557] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:00:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:00:11.642114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:11.643750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:11.658909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3105a362-4354-4491-a54e-55587a9ef931 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:11.664415 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:11.665040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:11.665390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:11.665686 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:11.667390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:11.679197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:11.681497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:11.691453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1288/2558] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:00:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:00:11.825633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:11.827039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:11.839174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c55f5c5-6f71-4e2e-93a7-65c9ce024c7f service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:11.843998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:11.844619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:11.844896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:11.845197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] PATH_INFO: `/v3/users/2ca578d6a1374212a4f1476335c641b3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:11.846299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ca578d6a1374212a4f1476335c641b3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:11.848204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:12.054674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ca578d6a1374212a4f1476335c641b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.055351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ca578d6a1374212a4f1476335c641b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.067680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ca578d6a1374212a4f1476335c641b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.068287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ca578d6a1374212a4f1476335c641b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.070447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:12.076529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-dd811352-ed45-4326-a223-66aaaa26ff0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ca578d6a1374212a4f1476335c641b3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.241185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1271/2559] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:00:11 2022] DELETE /identity/v3/users/2ca578d6a1374212a4f1476335c641b3 => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:00:12.532901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:12.545235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bbadd8-f980-41ef-b3fa-55bb4660dcb5 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:12.606550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:12.607104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:12.607385 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:12.607659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] PATH_INFO: `/v3/projects/0430b3e5fcd0415fb2c658443d8f6bf1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:12.608769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0430b3e5fcd0415fb2c658443d8f6bf1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:12.610697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:12.732016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0430b3e5fcd0415fb2c658443d8f6bf1', 'request_id': 'req-7a98a495-ab71-48ed-9f88-42f3baae6367'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.737660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0430b3e5fcd0415fb2c658443d8f6bf1', 'request_id': 'req-7a98a495-ab71-48ed-9f88-42f3baae6367'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.828224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0430b3e5fcd0415fb2c658443d8f6bf1', 'request_id': 'req-7a98a495-ab71-48ed-9f88-42f3baae6367'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.840036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] The token cache is being invalidate because project 0430b3e5fcd0415fb2c658443d8f6bf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:00:12.840329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.840968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-7a98a495-ab71-48ed-9f88-42f3baae6367 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:00:12.842514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1289/2560] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:00:12 2022] DELETE /identity/v3/projects/0430b3e5fcd0415fb2c658443d8f6bf1 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:00:12.848597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:12.849112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:12.849253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:12.911415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:00:12.912560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:12.971172 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1272/2561] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:00:12.979352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:12.979636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:12.979915 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.000865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:00:13.002087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:13.010705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1290/2562] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:00:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:00:13.014875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.025035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84893584-d4bb-416a-b4f2-2adf3cd389b3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.029845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.030437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.030711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.030977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.031617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:00:13.031914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.033524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.045476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1273/2563] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:00:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:00:13.049836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.060002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef8964d-808c-413f-b9e3-1702cde65bbb None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.064597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.065215 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.065494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.065761 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.066376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.068253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.196849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1291/2564] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:00:13 2022] POST /identity/v3/projects => generated 363 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:00:13.201264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.211450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9d48e-7781-425c-a2db-4d12c1a50e2f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.267101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.267705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.267981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.268248 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.268859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.270842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.383789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1274/2565] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:00:13 2022] POST /identity/v3/users => generated 282 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:00:13.388047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.399172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aeace7-f977-4b72-a83a-675d6981dd6f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.404212 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.404847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.405166 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.405438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.406111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.407688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.418887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1292/2566] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:00:13.422879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.432975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28023467-5cfd-47f8-87f0-b62f31fb9c18 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.437550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee455a92-adb2-44af-9c6c-c0a9725056cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.438168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee455a92-adb2-44af-9c6c-c0a9725056cb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.438445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee455a92-adb2-44af-9c6c-c0a9725056cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.438712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee455a92-adb2-44af-9c6c-c0a9725056cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.439349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee455a92-adb2-44af-9c6c-c0a9725056cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.441059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee455a92-adb2-44af-9c6c-c0a9725056cb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.451908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1275/2567] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:00:13 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:00:13.455934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.466204 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bee8fe-87ea-4dd3-9ee4-0dba5a38606b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.470597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.471229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.471509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.471779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] PATH_INFO: `/v3/projects/314279c4881d4363ac730631dd1aa1b3/users/41cab6e6382f4f0fae821dd321f4eefa/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.484050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:13.487123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=314279c4881d4363ac730631dd1aa1b3, user_id=41cab6e6382f4f0fae821dd321f4eefa, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.489796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.595064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1293/2568] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:00:13 2022] PUT /identity/v3/projects/314279c4881d4363ac730631dd1aa1b3/users/41cab6e6382f4f0fae821dd321f4eefa/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:00:13.606972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.607279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.607556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.635109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] MFA Rules not processed for user `41cab6e6382f4f0fae821dd321f4eefa`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:00:13.636303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:13.644497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:13.701671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1276/2569] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:00:13 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:00:13.706328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.766350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e0533-9798-4832-9a3e-b38b31c1bb43 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.770997 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.771589 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.771864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.772129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.772742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.775263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:13.892318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1294/2570] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:00:13 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:00:13.896463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:13.908038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa803f3-1b4d-4f52-9ed1-526805a422fe None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:13.964373 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:13.964893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:13.965201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:13.965478 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:13.966092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:13.967989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:14.079942 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1277/2571] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:00:13 2022] POST /identity/v3/users => generated 289 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:00:14.083740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:14.094976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eebba84-5597-4b9e-8068-a5b24f4f0fd4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:14.099580 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:14.100178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:14.100459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:14.100739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:14.101693 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:14.103333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:14.114107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1295/2572] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:00:14.118514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:14.128632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed18ec-b2ea-4123-809a-9cd2915bd5e4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:14.134149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:14.134837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:14.135118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:14.135392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] PATH_INFO: `/v3/projects/59380b72483c4dadbda01400a682d14d/users/9579e8ba16084bd3962442438c08a66a/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:14.155105 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=59380b72483c4dadbda01400a682d14d, user_id=9579e8ba16084bd3962442438c08a66a, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:14.157921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:14.265803 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1278/2573] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:00:14 2022] PUT /identity/v3/projects/59380b72483c4dadbda01400a682d14d/users/9579e8ba16084bd3962442438c08a66a/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:00:14.269896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:14.280788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdec65e-45ee-4bcd-993b-b41503f4a31f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:14.343168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966a8ae2-68c9-47d4-8cb6-08998ccd63b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:14.343783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a8ae2-68c9-47d4-8cb6-08998ccd63b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:14.344064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a8ae2-68c9-47d4-8cb6-08998ccd63b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:14.344339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966a8ae2-68c9-47d4-8cb6-08998ccd63b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:14.344996 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a8ae2-68c9-47d4-8cb6-08998ccd63b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:14.346619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966a8ae2-68c9-47d4-8cb6-08998ccd63b6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:14.357182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1296/2574] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:00:14 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:00:14.361562 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:14.372649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eb44b3-7300-4af5-81c1-c0b1f59f90cf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:14.377438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:14.378091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:14.378371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:14.378644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] PATH_INFO: `/v3/projects/59380b72483c4dadbda01400a682d14d/users/9579e8ba16084bd3962442438c08a66a/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:14.380561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59380b72483c4dadbda01400a682d14d, user_id=9579e8ba16084bd3962442438c08a66a, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:14.384290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:14.394779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1279/2575] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:00:14 2022] PUT /identity/v3/projects/59380b72483c4dadbda01400a682d14d/users/9579e8ba16084bd3962442438c08a66a/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:00:14.407477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:14.407746 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:14.408026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:14.436683 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] MFA Rules not processed for user `9579e8ba16084bd3962442438c08a66a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:00:14.437933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:14.489309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:14.513218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1297/2576] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:00:14 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:00:24.391008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=67,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:24.595000 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:00:27.770636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:00:27.772869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:27.884872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df7ec05-3ff5-41ae-b0cc-252704ce4867 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:00:27.890440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:00:27.891047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:00:27.891327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:00:27.891655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:00:27.893469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:00:27.906059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:00:27.907740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:00:27.918512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1280/2577] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:00:27 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:00:37.914679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=50 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:01:02.906956 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:01:02.908354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:01:02.966569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ab7667-4ae0-443a-99bb-4902f248c176 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:01:02.970250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:01:02.970875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:01:02.971156 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:01:02.971431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:01:02.980640 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:01:02.982113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:01:02.992357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1298/2578] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:01:02 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:01:03.303066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:01:03.315413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bcec0ef-6787-4ae1-be04-e6274ea85548 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:01:03.319490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:01:03.320618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:01:03.320973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:01:03.321294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:01:03.387481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:01:03.388829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:01:03.399609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1281/2579] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:01:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:01:04.580616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:01:04.582231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:01:04.658745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff449347-b6da-4716-bd99-461c8a04b05a service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:01:04.665744 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:01:04.672078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:01:04.672078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:01:04.672078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:01:04.672078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:01:04.682137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:01:04.683879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:01:04.694495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1299/2580] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:01:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:01:13.393795 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=28 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:01:14.690205 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=60 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:02:07.794180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:02:07.795649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:02:07.828367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3d10db-e250-48ac-8983-ea8b65d4aaca service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:02:07.831996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:02:07.832608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:02:07.832892 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:02:07.833201 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:02:07.835000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:02:07.908200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:02:07.909674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:02:07.928102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1282/2581] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:02:07 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:02:08.084055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:02:08.104836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e0fdbc-04da-46dc-b210-8672282e31fa service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:02:08.108439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:02:08.109065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:02:08.109344 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:02:08.109616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:02:08.111400 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:02:08.172320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:02:08.173771 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:02:08.183191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1300/2582] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:02:08 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:02:17.925466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=45 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:02:18.178659 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=33 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:02:24.512587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:02:24.514040 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:02:24.588532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59290a7e-3a0d-4ea9-9594-bab83627a0a0 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:02:24.592225 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:02:24.592806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:02:24.593121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:02:24.593389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:02:24.595153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:02:24.606412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:02:24.607837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:02:24.620178 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1283/2583] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:02:24 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:02:34.616868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=30 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:05:34.785576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:05:34.797667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a20a26-a895-48f5-a646-0f16c0f03fbb service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:05:34.802724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:05:34.803879 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:05:34.804157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:05:34.804433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:05:34.813421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:05:34.815003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:05:34.825789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1301/2584] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:05:34 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:05:44.821452 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:06:08.553533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:06:08.561398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ed5d10-419f-400b-a1e8-b53aba983156 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:06:08.565437 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:06:08.566136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:06:08.566466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:06:08.566820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:06:08.568893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:06:08.647731 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:06:08.649298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:06:08.670989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1284/2585] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:06:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:06:08.717633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:06:08.727431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb2eeab-95e8-45c7-93a9-d5bbfb9f1192 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:06:08.732436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:06:08.733088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:06:08.733377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:06:08.733656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:06:08.745520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:06:08.748242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:06:08.766287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1302/2586] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:06:08 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:06:08.881996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:06:08.892195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b38f35d-8631-4e91-9095-d2554ffcf815 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:06:08.898806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:06:08.898806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:06:08.898806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:06:08.898806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:06:08.899336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:06:09.030109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:06:09.032335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:06:09.067885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1285/2587] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:06:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:06:18.757724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:06:19.062956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=77 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:05.397668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:05.407888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04413019-7969-474e-a77e-015f93df3e9b service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:05.411861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:05.412479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:05.412760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:05.413052 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:05.424983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:05.426500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:05.439339 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1303/2588] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:05 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:07:05.811558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:05.822078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b93cb3-cf88-439b-9a0c-9840be8a3fb1 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:05.827169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:05.827822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:05.828104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:05.828378 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:05.830290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:05.903402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:05.905512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:05.917738 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1286/2589] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:09.950887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:09.964989 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1280e01e-6046-43c4-9f62-c01a1efb590b service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:09.971732 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:09.972485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:09.972832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:09.973208 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:09.975255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:09.989565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:09.994087 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:10.002520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1304/2590] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:10.111220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:10.114396 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:10.135745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58bc053-f533-4877-9a57-e7170caf77ae service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:10.144654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:10.145854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:10.146333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:10.146828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] PATH_INFO: `/v3/users/3eda1fca08b24727929b63318fd6f13e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:10.148975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eda1fca08b24727929b63318fd6f13e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:10.151301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:10.378188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eda1fca08b24727929b63318fd6f13e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:10.378866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eda1fca08b24727929b63318fd6f13e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:10.395243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eda1fca08b24727929b63318fd6f13e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:10.396119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eda1fca08b24727929b63318fd6f13e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:10.404475 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-31ece49e-4369-4058-8bfc-4aca5c7c21a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eda1fca08b24727929b63318fd6f13e'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:10.506666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1287/2591] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:07:10 2022] DELETE /identity/v3/users/3eda1fca08b24727929b63318fd6f13e => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:07:10.782377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:10.794207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa11060f-0387-47ff-8a53-f3d3480c4d85 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:10.855330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:10.855975 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:10.856247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:10.856517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] PATH_INFO: `/v3/projects/b02af2d33ca74bd08365dfc34a6fa77a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:10.858287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b02af2d33ca74bd08365dfc34a6fa77a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:10.860193 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:11.012182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02af2d33ca74bd08365dfc34a6fa77a', 'request_id': 'req-4e845f71-454d-48ea-9b2e-40a9650f164f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:11.017818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02af2d33ca74bd08365dfc34a6fa77a', 'request_id': 'req-4e845f71-454d-48ea-9b2e-40a9650f164f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:11.117282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02af2d33ca74bd08365dfc34a6fa77a', 'request_id': 'req-4e845f71-454d-48ea-9b2e-40a9650f164f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:11.130114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] The token cache is being invalidate because project b02af2d33ca74bd08365dfc34a6fa77a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:07:11.130442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:11.131114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4e845f71-454d-48ea-9b2e-40a9650f164f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:07:11.132597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1305/2592] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:07:10 2022] DELETE /identity/v3/projects/b02af2d33ca74bd08365dfc34a6fa77a => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:07:11.139428 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.139956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.140099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.163698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:07:11.166254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:11.370044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1288/2593] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:07:11.378121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.378457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.378814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.399327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:07:11.400540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:11.410222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1306/2594] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:07:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:07:11.417126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:11.435553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1959613-5e6d-4ff9-933b-32b3ecd39695 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:11.442969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:11.444016 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.444502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.444985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.446584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:07:11.447135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:11.450139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:11.453587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:11.472872 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1289/2595] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:07:11.477733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:11.489372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beca89ef-e440-4941-8a9d-4cabf4a25342 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:11.494345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:11.494972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.495250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.495523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.496275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:11.498751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:11.682421 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1307/2596] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:07:11 2022] POST /identity/v3/projects => generated 385 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:07:11.689424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:11.706936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861f19cd-14de-44b1-9e6d-0d94c2963097 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:11.785863 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:11.786486 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.786769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.787042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.787673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:11.789740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:11.876908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1290/2597] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:07:11 2022] POST /identity/v3/users => generated 293 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:07:11.881237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:11.892142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d42a49-74dc-484e-8cde-43f503dea46d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:11.897185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:11.897792 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.898073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.898347 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.899073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:11.900668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:11.912149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1308/2598] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:07:11.916332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:11.928216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88cf796-fbe7-4e8f-9646-b4022a62f414 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:11.933355 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da14109b-36ca-4aab-8ff3-ce04e2e99764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:11.933908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14109b-36ca-4aab-8ff3-ce04e2e99764 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.934196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14109b-36ca-4aab-8ff3-ce04e2e99764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.934492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da14109b-36ca-4aab-8ff3-ce04e2e99764 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.935762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14109b-36ca-4aab-8ff3-ce04e2e99764 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:11.937593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da14109b-36ca-4aab-8ff3-ce04e2e99764 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:11.949319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1291/2599] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:07:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:07:11.954796 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:11.965306 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224598ff-9382-4443-887b-6b334f32113c admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:11.970889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:11.971557 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:11.972130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:11.972130 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] PATH_INFO: `/v3/projects/226fb1c97e3f4e87a47326dae7fb262d/users/b7d2ab8bd072488091fb5a94f59946d3/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:11.992926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] RBAC: Authorizing `identity:create_grant(project_id=226fb1c97e3f4e87a47326dae7fb262d, user_id=b7d2ab8bd072488091fb5a94f59946d3, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:11.995636 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:12.100272 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:12.105564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1309/2600] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:07:11 2022] PUT /identity/v3/projects/226fb1c97e3f4e87a47326dae7fb262d/users/b7d2ab8bd072488091fb5a94f59946d3/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:07:12.111692 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19655988-f60d-4add-8c7b-f49859acc832 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:12.111962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19655988-f60d-4add-8c7b-f49859acc832 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:12.112227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19655988-f60d-4add-8c7b-f49859acc832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:12.132354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-19655988-f60d-4add-8c7b-f49859acc832 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:07:12.133595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-19655988-f60d-4add-8c7b-f49859acc832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:12.198425 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1292/2601] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:07:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:07:12.210760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:12.213443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:12.299360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4ae8dd-42df-43c9-be60-5b9aa682833b admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:12.303049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:12.303666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:12.303944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:12.304216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:12.314504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:12.315940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:12.326799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1310/2602] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:18.922777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e0e5-9d0c-4ada-9cd9-62caa2f5566f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:18.923295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e0e5-9d0c-4ada-9cd9-62caa2f5566f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:18.923903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e188e0e5-9d0c-4ada-9cd9-62caa2f5566f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:18.971015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e188e0e5-9d0c-4ada-9cd9-62caa2f5566f None None] MFA Rules not processed for user `b7d2ab8bd072488091fb5a94f59946d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:07:18.972333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e188e0e5-9d0c-4ada-9cd9-62caa2f5566f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:19.067061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1293/2603] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:07:18 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:07:19.090837 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:19.097634 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:19.246145 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075c1443-d192-4176-8923-ef51d69116b5 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:19.253620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:19.254258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:19.254538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:19.254808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:19.269500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:19.272860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:19.290329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1311/2604] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 201 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:22.132238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:22.132878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:22.132878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:22.140286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:22.153258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] MFA Rules not processed for user `b7d2ab8bd072488091fb5a94f59946d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:07:22.154747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:22.162166 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1294/2605] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:07:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:07:22.213305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:22.215209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:22.310311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=100 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:22.325100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eaba3cf-0578-49f2-aba4-5a9122f32c67 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:22.332703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:22.333739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:22.334235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:22.334704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:22.339101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:22.359013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:22.361895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:22.377965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1312/2606] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:22 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:22.729128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:22.731657 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:22.782644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78463cff-c94a-4563-8ccc-39d5616b8e8d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:22.791191 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:22.792274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:22.792762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:22.793295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:22.804473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:22.817905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:22.819565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:22.830652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1295/2607] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:23.394259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:23.398233 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:23.417194 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:23.430070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd9c896-1ddb-4895-9682-ca33b6810d60 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:23.437409 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:23.438354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:23.438857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:23.439351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:23.442629 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:23.475176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:23.478173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:23.495500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1296/2608] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:23 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:23.546681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163ccf3-3263-44ce-9480-57717c017026 service glance] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:23.561330 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:23.563165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:23.563165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:23.563633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:23.566111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:23.585697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:23.588911 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:23.602677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1313/2609] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:27.764756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:27.777186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbbcb25-8b91-4efa-93be-33cbdded0225 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:27.782315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:27.783003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:27.783414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:27.783820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:27.785972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:27.798754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:27.800578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:27.812077 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1297/2610] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:27 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:07:28.677532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:28.683176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:28.698422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91027044-4f68-4029-b5d8-ded96e431e22 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:28.705685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:28.706556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:28.706946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:28.707428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:28.709766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:28.729137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:28.730566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:28.744144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1314/2611] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:07:35.661096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:35.681561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381e76e5-9b71-4ec8-b5d6-10ff30ec7cfa service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:35.687993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:35.688724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:35.689133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:35.689467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:35.691481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:35.706319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:35.707977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:35.721344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1298/2612] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:07:38.738501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=45 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:40.633678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:07:40.663358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c727bf-ec64-43d6-8226-5a3d04b65578 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:07:40.676575 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:07:40.678073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:07:40.678543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:07:40.678966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:07:40.682022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:07:40.702735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:07:40.704539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:07:40.721327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1315/2613] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:07:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:07:45.715308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=31 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:07:50.713585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:09.382190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:09.383647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:09.451498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed519287-b703-44c7-85f9-471766ea6a5e service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:09.455208 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:09.455814 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:09.456090 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:09.456361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:09.458147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:09.537231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:09.538669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:09.550525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1299/2614] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:09 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:09.704569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:09.714192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acaf37eb-b268-4dfe-ab58-e372c9193f55 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:09.717919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:09.718547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:09.719429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:09.719819 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:09.721748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:09.806288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:09.807947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c088ce7f-fb29-4242-a62f-a5c42967baf3 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:09.831271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1316/2615] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:09 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:19.543649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=58 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:19.828067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=41 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:36.737481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:36.738912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:36.753101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524c8256-a938-4dae-aa26-0032767a63a1 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:36.757616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:36.758235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:36.758507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:36.758772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] PATH_INFO: `/v3/users/41cab6e6382f4f0fae821dd321f4eefa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:36.759882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41cab6e6382f4f0fae821dd321f4eefa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:36.761807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:36.959315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41cab6e6382f4f0fae821dd321f4eefa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:36.959991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41cab6e6382f4f0fae821dd321f4eefa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:36.973187 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41cab6e6382f4f0fae821dd321f4eefa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:36.980076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41cab6e6382f4f0fae821dd321f4eefa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:36.980723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41cab6e6382f4f0fae821dd321f4eefa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:37.053410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1300/2616] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:36 2022] DELETE /identity/v3/users/41cab6e6382f4f0fae821dd321f4eefa => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:37.058297 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:37.058762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:37.058852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:37.078012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:37.079714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:37.140169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1317/2617] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:37.150844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:37.152269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:37.211839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e34e63-d132-48ce-bf3b-a7e967af2395 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:37.215541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:37.216152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:37.216430 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:37.216724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:37.226038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:37.227438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:37.237320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1301/2618] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:37.640738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:37.643420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:37.656656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ac0b30-0cce-4a96-99c3-3bf955772184 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:37.668367 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:37.669357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:37.669847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:37.670335 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] PATH_INFO: `/v3/projects/314279c4881d4363ac730631dd1aa1b3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:37.672009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] RBAC: Authorizing `identity:delete_project(project_id=314279c4881d4363ac730631dd1aa1b3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:37.675399 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:37.791918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '314279c4881d4363ac730631dd1aa1b3', 'request_id': 'req-bcf35271-8645-46b9-ae4f-f2c1a0f82075'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:37.799325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '314279c4881d4363ac730631dd1aa1b3', 'request_id': 'req-bcf35271-8645-46b9-ae4f-f2c1a0f82075'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:37.841337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '314279c4881d4363ac730631dd1aa1b3', 'request_id': 'req-bcf35271-8645-46b9-ae4f-f2c1a0f82075'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:37.856024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] The token cache is being invalidate because project 314279c4881d4363ac730631dd1aa1b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:37.856299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:37.856998 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:37.858855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1318/2619] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:37 2022] DELETE /identity/v3/projects/314279c4881d4363ac730631dd1aa1b3 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:37.863653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:37.875665 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25005aa9-de9d-4d28-b59b-a5a4ff3468de service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:37.935085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:37.935719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:37.936014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:37.936369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] PATH_INFO: `/v3/users/9579e8ba16084bd3962442438c08a66a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:37.937518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=9579e8ba16084bd3962442438c08a66a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:37.940171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:37.956449 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:38.113174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9579e8ba16084bd3962442438c08a66a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.113779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9579e8ba16084bd3962442438c08a66a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.125952 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9579e8ba16084bd3962442438c08a66a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.127421 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9579e8ba16084bd3962442438c08a66a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.138243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-93a39e25-5a4b-4a4d-9647-71859849d4fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9579e8ba16084bd3962442438c08a66a'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.213579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1302/2620] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:37 2022] DELETE /identity/v3/users/9579e8ba16084bd3962442438c08a66a => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:38.682929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:38.684423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:38.697899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf35271-8645-46b9-ae4f-f2c1a0f82075 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:38.756106 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:38.762303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:38.762303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:38.762303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] PATH_INFO: `/v3/projects/59380b72483c4dadbda01400a682d14d` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:38.762303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59380b72483c4dadbda01400a682d14d)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:38.762303 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:38.834119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:38.853653 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59380b72483c4dadbda01400a682d14d', 'request_id': 'req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.861070 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59380b72483c4dadbda01400a682d14d', 'request_id': 'req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.939152 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59380b72483c4dadbda01400a682d14d', 'request_id': 'req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.952851 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] The token cache is being invalidate because project 59380b72483c4dadbda01400a682d14d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:38.953555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.955002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-e4bab1e5-7a7d-48c8-a67b-7f60e2aa8651 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:38.957308 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1319/2621] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:38 2022] DELETE /identity/v3/projects/59380b72483c4dadbda01400a682d14d => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:38.962513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:38.962882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:38.963023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:38.991265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:38.992494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:39.064472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1303/2622] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:39.072502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.072779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.073084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.095739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:39.096977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:39.110437 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1320/2623] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:39.115232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.128579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dafdd8d-d332-4aca-926f-9bd7dbf7d497 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.133443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.134058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.134335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.134613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.135268 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:08:39.135567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.137559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:39.153249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1304/2624] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:08:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:08:39.159211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.170076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5e3346-85fd-4d96-807e-ddc8b13b329f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.175379 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.176002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.176284 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.176608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.177280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.179188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:39.335871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1321/2625] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:39 2022] POST /identity/v3/projects => generated 385 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:39.340097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.352623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd011c8-aca8-4ae8-94ac-01dfe7081cc3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.421333 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.421963 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.422244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.422545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.424381 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.427274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:39.510725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1305/2626] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:39 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:39.515162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.527962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aacb5c2-2799-42e6-89d6-687edc34615e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.535567 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.536345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.536800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.537142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.538137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.540315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:39.558376 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1322/2627] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:39.560076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.583518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef638240-00ae-4774-bb32-71ad69b59460 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.590310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b1ff1f-723b-4d47-9e8f-602962857f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.590851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1ff1f-723b-4d47-9e8f-602962857f34 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.591126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1ff1f-723b-4d47-9e8f-602962857f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.591393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b1ff1f-723b-4d47-9e8f-602962857f34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.592038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1ff1f-723b-4d47-9e8f-602962857f34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.593672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b1ff1f-723b-4d47-9e8f-602962857f34 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:39.602620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:39.609092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1306/2628] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:39 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:39.620259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.631779 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51414df4-c11e-478e-9474-c8f9a3d25d06 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.637435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.638206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.638491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.638763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] PATH_INFO: `/v3/projects/7a7141ffa8f045ee80b33b4faec1d31a/users/33aba55f68b049e0ab40f5e34ad3fafb/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.655260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a7141ffa8f045ee80b33b4faec1d31a, user_id=33aba55f68b049e0ab40f5e34ad3fafb, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.658682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:39.716150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1323/2629] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:39 2022] PUT /identity/v3/projects/7a7141ffa8f045ee80b33b4faec1d31a/users/33aba55f68b049e0ab40f5e34ad3fafb/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:39.727165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.727446 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.727717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.756763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] MFA Rules not processed for user `33aba55f68b049e0ab40f5e34ad3fafb`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:39.758022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:39.822402 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1307/2630] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:39.827100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:39.889992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a492f-2720-4f37-9ec4-a4e4bb09e2e9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:39.895015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:39.895633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:39.895917 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:39.896188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:39.896865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:39.898789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.026381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1324/2631] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:39 2022] POST /identity/v3/projects => generated 387 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:40.030614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.041822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b271386-3e47-48ae-b49e-0948f6e0912c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.101159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.101765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.102041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.102309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.102944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.104831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.220260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1308/2632] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:40 2022] POST /identity/v3/users => generated 300 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:40.222523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.235224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9e6d8c-24ad-452a-927d-592c9cff91c0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.241669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.242327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.242609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.242883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.243539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.245457 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.264843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1325/2633] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:40.267689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.279114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ae7745-8257-4493-97b0-8f69ef5624e8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.284537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.287711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.287711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.287711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] PATH_INFO: `/v3/projects/cc7bfaa05cec4e15a0681455e83bc473/users/e7cf760559454164927c92441be377f6/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.306252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7bfaa05cec4e15a0681455e83bc473, user_id=e7cf760559454164927c92441be377f6, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.308641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.362155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1309/2634] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:40 2022] PUT /identity/v3/projects/cc7bfaa05cec4e15a0681455e83bc473/users/e7cf760559454164927c92441be377f6/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:40.366207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.376406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f1a71-123a-4a01-953a-83f1a53071c0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.397560 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:40.435721 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bc3c3e-ebdb-4b30-8bbc-1516fce6443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.436334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3c3e-ebdb-4b30-8bbc-1516fce6443f admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.436614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3c3e-ebdb-4b30-8bbc-1516fce6443f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.436888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bc3c3e-ebdb-4b30-8bbc-1516fce6443f admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.437568 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3c3e-ebdb-4b30-8bbc-1516fce6443f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.439527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bc3c3e-ebdb-4b30-8bbc-1516fce6443f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.450230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1326/2635] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:40.454940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.465469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7b0b19-aa22-41ed-87c8-fa453de236aa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.470785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.471920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.472401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.472869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] PATH_INFO: `/v3/projects/cc7bfaa05cec4e15a0681455e83bc473/users/e7cf760559454164927c92441be377f6/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.475750 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc7bfaa05cec4e15a0681455e83bc473, user_id=e7cf760559454164927c92441be377f6, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.479651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.492455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1310/2636] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:40 2022] PUT /identity/v3/projects/cc7bfaa05cec4e15a0681455e83bc473/users/e7cf760559454164927c92441be377f6/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:40.508496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.508955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.509279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.539844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] MFA Rules not processed for user `e7cf760559454164927c92441be377f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:40.541157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:40.614499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1327/2637] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:40 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:40.625820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.627553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:40.675857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:40.686879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ec997-d0c5-4adc-96ed-f8d2fcb871ca admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.690613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e19756-853a-4378-8944-c383c7170066 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.691252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e19756-853a-4378-8944-c383c7170066 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.691529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e19756-853a-4378-8944-c383c7170066 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.691793 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e19756-853a-4378-8944-c383c7170066 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.700579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e19756-853a-4378-8944-c383c7170066 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.704423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e19756-853a-4378-8944-c383c7170066 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:40.711265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1311/2638] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:40 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:40.778908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:40.842099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbf20bb-1ba4-43e8-8ff3-d94f26d6b1be None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:40.847405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:40.848037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:40.848310 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:40.848576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] PATH_INFO: `/v3/users/33aba55f68b049e0ab40f5e34ad3fafb` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:40.849716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33aba55f68b049e0ab40f5e34ad3fafb)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:40.851604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:41.074931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33aba55f68b049e0ab40f5e34ad3fafb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:41.075600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33aba55f68b049e0ab40f5e34ad3fafb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:41.076207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33aba55f68b049e0ab40f5e34ad3fafb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:41.077490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:41.084888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33aba55f68b049e0ab40f5e34ad3fafb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:41.099179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33aba55f68b049e0ab40f5e34ad3fafb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:41.225118 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1328/2639] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:40 2022] DELETE /identity/v3/users/33aba55f68b049e0ab40f5e34ad3fafb => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:41.229645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:41.230137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:41.230323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:41.250059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:41.251830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:41.318866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1312/2640] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:41.328884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:41.330961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:41.396463 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a6f1d3-bea7-4287-a599-c029a6866975 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:41.400496 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:41.401136 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:41.401411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:41.401680 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:41.413954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:41.415748 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:41.428969 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1329/2641] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:42.018026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:42.018026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:42.036026 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b34728-679c-4c7c-b2a8-c1f719ea3c01 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:42.041325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:42.041932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:42.042258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:42.042531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] PATH_INFO: `/v3/projects/7a7141ffa8f045ee80b33b4faec1d31a` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:42.043783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a7141ffa8f045ee80b33b4faec1d31a)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:42.045781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:42.224527 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a7141ffa8f045ee80b33b4faec1d31a', 'request_id': 'req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.231058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a7141ffa8f045ee80b33b4faec1d31a', 'request_id': 'req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.238573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a7141ffa8f045ee80b33b4faec1d31a', 'request_id': 'req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.343837 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] The token cache is being invalidate because project 7a7141ffa8f045ee80b33b4faec1d31a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:42.344133 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.350654 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.355234 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1313/2642] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:42 2022] DELETE /identity/v3/projects/7a7141ffa8f045ee80b33b4faec1d31a => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:42.356984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:42.370006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccf724-e204-4f40-ac6f-5e408b32d557 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:42.440841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:42.441564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:42.441842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:42.442119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] PATH_INFO: `/v3/users/e7cf760559454164927c92441be377f6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:42.443374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7cf760559454164927c92441be377f6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:42.445390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:42.640870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7cf760559454164927c92441be377f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.642469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7cf760559454164927c92441be377f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.643750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7cf760559454164927c92441be377f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.655769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7cf760559454164927c92441be377f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.670718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:42.676858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-4dc8e63b-e847-4cec-960c-ae059023380c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7cf760559454164927c92441be377f6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:42.769318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1330/2643] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:42 2022] DELETE /identity/v3/users/e7cf760559454164927c92441be377f6 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:43.254404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:43.256039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:43.272270 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6394c8bf-c30e-408d-8dc7-7a334a7e1f58 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:43.341335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:43.341988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:43.342908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:43.343366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] PATH_INFO: `/v3/projects/cc7bfaa05cec4e15a0681455e83bc473` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:43.344614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc7bfaa05cec4e15a0681455e83bc473)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:43.347641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:43.360708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:43.483320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc7bfaa05cec4e15a0681455e83bc473', 'request_id': 'req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:43.489828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc7bfaa05cec4e15a0681455e83bc473', 'request_id': 'req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:43.496387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc7bfaa05cec4e15a0681455e83bc473', 'request_id': 'req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:43.561664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] The token cache is being invalidate because project cc7bfaa05cec4e15a0681455e83bc473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:43.561973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:43.562599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-4336bc2b-8eca-4bba-affd-d35f9dc94fdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:43.564806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1314/2644] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:43 2022] DELETE /identity/v3/projects/cc7bfaa05cec4e15a0681455e83bc473 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:43.569934 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:43.570388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:43.570499 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:43.591705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:43.592902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:43.655373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1331/2645] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:43.663723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:43.666335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:43.666335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:43.684176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:43.685455 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:43.697254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1315/2646] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:43.701383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:43.712340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d9efe4-766d-4907-81bb-d6844a8139be None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:43.717587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:43.718666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:43.719017 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:43.719358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:43.720490 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:08:43.720797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:43.722778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:43.739887 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1332/2647] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:08:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 23:08:43.743318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:43.753438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8c5c48-0f29-4a20-8d46-a4cb0e75f925 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:43.758047 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:43.758638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:43.758912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:43.759260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:43.759895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:43.761825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:43.877670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1316/2648] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:43 2022] POST /identity/v3/projects => generated 389 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:43.877902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:43.887857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffcfad5-21c1-446d-8d16-7bc74dd0bdb3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:43.948478 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:43.949702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:43.949986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:43.950258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:43.950900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:43.952822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.093538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1333/2649] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:43 2022] POST /identity/v3/users => generated 295 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:44.097640 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.108423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6894ebbb-7eff-4e21-b227-dd26c21fbe6b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.113471 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.114098 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.114419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.114623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.115265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.116852 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.128036 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1317/2650] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:44.132381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.142599 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba1e86-4188-4af4-884a-857445c860a6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.147396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84439c05-b3f5-423a-901c-550fcc4dc993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.147995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439c05-b3f5-423a-901c-550fcc4dc993 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.148280 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439c05-b3f5-423a-901c-550fcc4dc993 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.148648 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84439c05-b3f5-423a-901c-550fcc4dc993 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.149340 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84439c05-b3f5-423a-901c-550fcc4dc993 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.151203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84439c05-b3f5-423a-901c-550fcc4dc993 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.162247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1334/2651] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:44.166616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.177392 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6814004-edae-4bc0-b50e-9aedb70cbeb8 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.182017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.183332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.183613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.183892 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] PATH_INFO: `/v3/projects/1b23ee616ad5407885252d477d86a6da/users/bb022c61a2a0462a9a38d20eb69d68e5/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.200462 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] RBAC: Authorizing `identity:create_grant(project_id=1b23ee616ad5407885252d477d86a6da, user_id=bb022c61a2a0462a9a38d20eb69d68e5, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.203038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.299693 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1318/2652] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:44 2022] PUT /identity/v3/projects/1b23ee616ad5407885252d477d86a6da/users/bb022c61a2a0462a9a38d20eb69d68e5/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:44.310481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.310770 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.311453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.339352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] MFA Rules not processed for user `bb022c61a2a0462a9a38d20eb69d68e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:44.340562 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:44.384141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:44.407183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1335/2653] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:44 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:44.412232 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.478041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4116617-372d-44f6-b5e4-71d2e32a553a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.482908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.483599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.483882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.484156 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.484785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.486794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.558112 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1319/2654] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:44 2022] POST /identity/v3/projects => generated 389 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:44.562256 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.573245 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa7fcc-7861-4f1b-903f-36700814e3aa None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.630046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.630657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.630944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.631219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.631839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.633807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.711190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1336/2655] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:44 2022] POST /identity/v3/users => generated 301 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:44.715171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.726463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad7c83c-717a-4b3a-91df-6a0e03b81c7c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.731106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.731644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.732030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.732306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.732932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.734551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.746206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1320/2656] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:44.750462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.763329 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75218cd-9efd-4701-8c82-6eb1b6aace6f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.769113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.769781 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.770062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.770346 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] PATH_INFO: `/v3/projects/df15fe8423464d5496f5275639e1cbd3/users/f035aebb518e46a798293521df1bb20f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.794001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df15fe8423464d5496f5275639e1cbd3, user_id=f035aebb518e46a798293521df1bb20f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.796702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.853397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1337/2657] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:44 2022] PUT /identity/v3/projects/df15fe8423464d5496f5275639e1cbd3/users/f035aebb518e46a798293521df1bb20f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:44.857639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.868151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1955d436-da26-4a50-8b42-b5a36b6389fe admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.872789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:44.948503 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e67a2ed-a814-4423-8f49-9e217f4e251e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.949258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67a2ed-a814-4423-8f49-9e217f4e251e admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.949552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67a2ed-a814-4423-8f49-9e217f4e251e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.949966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e67a2ed-a814-4423-8f49-9e217f4e251e admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.950628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67a2ed-a814-4423-8f49-9e217f4e251e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.952295 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e67a2ed-a814-4423-8f49-9e217f4e251e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:44.965340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1321/2658] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:44.969935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:44.987264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1864862-ba62-4149-bb94-9feaf8912e24 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:44.991197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:44.991865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:44.992148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:44.992428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] PATH_INFO: `/v3/projects/df15fe8423464d5496f5275639e1cbd3/users/f035aebb518e46a798293521df1bb20f/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:44.994500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=df15fe8423464d5496f5275639e1cbd3, user_id=f035aebb518e46a798293521df1bb20f, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:44.997176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:45.009121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1338/2659] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:44 2022] PUT /identity/v3/projects/df15fe8423464d5496f5275639e1cbd3/users/f035aebb518e46a798293521df1bb20f/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:45.078906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:45.079188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:45.079458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:45.108936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] MFA Rules not processed for user `f035aebb518e46a798293521df1bb20f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:45.110243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:45.179342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1322/2660] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:45 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:45.192319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:45.194180 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:45.254931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a3a6f2-73b9-4959-93c0-41eca6bc65fb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:45.258826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34af16ef-9099-49f7-86c5-4365ebaccd6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:45.259427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af16ef-9099-49f7-86c5-4365ebaccd6d service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:45.259706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af16ef-9099-49f7-86c5-4365ebaccd6d service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:45.260011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34af16ef-9099-49f7-86c5-4365ebaccd6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:45.269427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af16ef-9099-49f7-86c5-4365ebaccd6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:45.270846 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34af16ef-9099-49f7-86c5-4365ebaccd6d service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:45.284096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1339/2661] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:45 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:45.435906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:45.503752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7eb85b-5b16-4f3f-b071-d762d24e82b3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:45.508249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:45.508817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:45.509129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:45.509401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] PATH_INFO: `/v3/users/bb022c61a2a0462a9a38d20eb69d68e5` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:45.510515 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb022c61a2a0462a9a38d20eb69d68e5)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:45.512410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:45.694831 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:45.708153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb022c61a2a0462a9a38d20eb69d68e5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:45.709336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb022c61a2a0462a9a38d20eb69d68e5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:45.727053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb022c61a2a0462a9a38d20eb69d68e5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:45.727799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb022c61a2a0462a9a38d20eb69d68e5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:45.735636 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb022c61a2a0462a9a38d20eb69d68e5'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:45.822408 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1323/2662] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:45 2022] DELETE /identity/v3/users/bb022c61a2a0462a9a38d20eb69d68e5 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:45.827543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:45.828163 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:45.830551 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:45.848514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:45.849757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:45.882991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:45.911120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1340/2663] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:45.922102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:45.924347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:46.008717 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6872291-3181-46a8-a8f5-4a5e881f12b1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:46.012420 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:46.013057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:46.013337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:46.013610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:46.024044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:46.025681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:46.035557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1324/2664] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:46.489113 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:46.490619 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:46.507504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c0dff2-ebb5-4eb8-97ec-fa629afdc442 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:46.527289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:46.528109 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:46.528982 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:46.529443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] PATH_INFO: `/v3/projects/1b23ee616ad5407885252d477d86a6da` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:46.530727 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1b23ee616ad5407885252d477d86a6da)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:46.533553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:46.672455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1b23ee616ad5407885252d477d86a6da', 'request_id': 'req-200aa90a-88cc-4132-b054-4edfb528ebd3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:46.681730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1b23ee616ad5407885252d477d86a6da', 'request_id': 'req-200aa90a-88cc-4132-b054-4edfb528ebd3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:46.750537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1b23ee616ad5407885252d477d86a6da', 'request_id': 'req-200aa90a-88cc-4132-b054-4edfb528ebd3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:46.763203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] The token cache is being invalidate because project 1b23ee616ad5407885252d477d86a6da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:46.763495 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:46.764394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:46.766112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1341/2665] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:46 2022] DELETE /identity/v3/projects/1b23ee616ad5407885252d477d86a6da => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:46.770754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:46.787972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404fe9b1-0a83-4802-932c-81dad6a243e7 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:46.850151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:46.850789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:46.851066 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:46.851337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] PATH_INFO: `/v3/users/f035aebb518e46a798293521df1bb20f` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:46.852470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f035aebb518e46a798293521df1bb20f)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:46.854412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:47.074808 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f035aebb518e46a798293521df1bb20f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.075574 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f035aebb518e46a798293521df1bb20f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.082070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:47.090895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f035aebb518e46a798293521df1bb20f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.091653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f035aebb518e46a798293521df1bb20f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.100006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-557e5306-db16-465d-9b84-df19512d8dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f035aebb518e46a798293521df1bb20f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.207282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1325/2666] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:46 2022] DELETE /identity/v3/users/f035aebb518e46a798293521df1bb20f => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:47.632871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:47.634403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:47.648806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200aa90a-88cc-4132-b054-4edfb528ebd3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:47.735191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:47.737547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:47.737898 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:47.738287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] PATH_INFO: `/v3/projects/df15fe8423464d5496f5275639e1cbd3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:47.740574 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df15fe8423464d5496f5275639e1cbd3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:47.742755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:47.881169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df15fe8423464d5496f5275639e1cbd3', 'request_id': 'req-111f4e05-c6ca-499f-842f-9c31eba97295'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.886661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df15fe8423464d5496f5275639e1cbd3', 'request_id': 'req-111f4e05-c6ca-499f-842f-9c31eba97295'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.890405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:47.961943 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df15fe8423464d5496f5275639e1cbd3', 'request_id': 'req-111f4e05-c6ca-499f-842f-9c31eba97295'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.973977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] The token cache is being invalidate because project df15fe8423464d5496f5275639e1cbd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:47.974270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.974909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-111f4e05-c6ca-499f-842f-9c31eba97295 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:47.976428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1342/2667] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:47 2022] DELETE /identity/v3/projects/df15fe8423464d5496f5275639e1cbd3 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:47.981528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:47.981980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:47.982093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.003390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:48.004960 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:48.066533 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1326/2668] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:48.074242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60163541-4abb-468a-8f74-fc8607083ece None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.076581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60163541-4abb-468a-8f74-fc8607083ece None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.076581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60163541-4abb-468a-8f74-fc8607083ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.094285 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-60163541-4abb-468a-8f74-fc8607083ece None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:48.096132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-60163541-4abb-468a-8f74-fc8607083ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:48.105148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1343/2669] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:48.109543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.120434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4939b1-e291-422d-b710-70a4bdcb53dc None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.125357 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.126048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.126330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.126745 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.127405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:08:48.127714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.129372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.141311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1327/2670] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:08:48.145946 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60163541-4abb-468a-8f74-fc8607083ece None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.156597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60163541-4abb-468a-8f74-fc8607083ece None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.163873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.164504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.164786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.165088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.165734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.167685 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.238273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1344/2671] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:48 2022] POST /identity/v3/projects => generated 385 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:48.242456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.252930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d353684-b2f3-4524-adec-bc17043274c0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.311386 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.311990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.312272 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.312547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.313197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.315397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.407467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1328/2672] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:48 2022] POST /identity/v3/users => generated 293 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:48.412476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.422824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee984cc2-267a-4abc-83e9-a48211d54eb5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.427600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.428182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.428462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.428736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.429440 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.431033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.441454 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1345/2673] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:48.445506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.456200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792d7d9d-f8ea-4927-a9ba-93135c0deda1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.460943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f377ccd4-8166-435e-992c-c42f60714334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.461568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ccd4-8166-435e-992c-c42f60714334 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.461848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ccd4-8166-435e-992c-c42f60714334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.462124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f377ccd4-8166-435e-992c-c42f60714334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.462765 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377ccd4-8166-435e-992c-c42f60714334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.464335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f377ccd4-8166-435e-992c-c42f60714334 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.476675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1329/2674] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:48.480892 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.491294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dcf2ca-2937-469d-a67d-5843749dcd9e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.495950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.496621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.496891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.497202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] PATH_INFO: `/v3/projects/5b786d3c743d49088b5ac3d72413d3fa/users/3e9bf589176c4889b1573aef4fa7861c/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.512252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b786d3c743d49088b5ac3d72413d3fa, user_id=3e9bf589176c4889b1573aef4fa7861c, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.514940 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.603427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1346/2675] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:48 2022] PUT /identity/v3/projects/5b786d3c743d49088b5ac3d72413d3fa/users/3e9bf589176c4889b1573aef4fa7861c/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:48.614659 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.614943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.615216 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.652655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] MFA Rules not processed for user `3e9bf589176c4889b1573aef4fa7861c`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:48.654788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:48.687269 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:48.742243 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1330/2676] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:48.748696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.824538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552726cc-5125-48ec-baae-674ba5d9d2b7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.829696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.831059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.831217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.831403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.832049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.833973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.862374 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.863900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:48.925053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0cef85-eb2e-4b56-b52b-9c2685fd43de None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:48.928568 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:48.929218 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:48.929497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:48.929772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:48.931552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:48.944485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:48.945944 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:48.956517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1331/2677] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:08:48.973054 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1347/2678] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:48 2022] POST /identity/v3/projects => generated 387 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:48.974576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:48.984052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2622d5-6bb6-42d4-828c-22e23d9a2d39 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:49.043352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:49.044331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.044627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.044920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.045689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:49.047784 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:49.143086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1332/2679] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:48 2022] POST /identity/v3/users => generated 300 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:49.144509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:49.155839 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a9f24-4f40-47e1-846f-47e2bab37409 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:49.160790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:49.161458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.161735 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.162011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.162666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:49.164270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:49.177832 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1348/2680] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:08:49.181463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:49.192583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb459dc0-58ce-4460-bd0b-3adde8642391 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:49.202538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:49.202538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.202538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.202538 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] PATH_INFO: `/v3/projects/2d066f71973b436fb46492fc86196f73/users/508b7eb0e9934216932bb1a1fc68aade/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.218786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d066f71973b436fb46492fc86196f73, user_id=508b7eb0e9934216932bb1a1fc68aade, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:49.222254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:49.288180 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1333/2681] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:49 2022] PUT /identity/v3/projects/2d066f71973b436fb46492fc86196f73/users/508b7eb0e9934216932bb1a1fc68aade/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:49.292323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:49.303857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577ba610-a34f-4937-9b06-080f16e717b5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:49.395076 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede56a34-d213-40aa-8caf-c980e9cc592e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:49.396058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56a34-d213-40aa-8caf-c980e9cc592e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.396485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56a34-d213-40aa-8caf-c980e9cc592e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.396980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede56a34-d213-40aa-8caf-c980e9cc592e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.398217 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede56a34-d213-40aa-8caf-c980e9cc592e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:49.401430 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede56a34-d213-40aa-8caf-c980e9cc592e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:49.404916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:49.416191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1349/2682] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:49.420522 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:49.431705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae0bcb4-c674-479c-b511-253d9a2b0292 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:49.436370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:49.437061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.437406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.437716 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] PATH_INFO: `/v3/projects/2d066f71973b436fb46492fc86196f73/users/508b7eb0e9934216932bb1a1fc68aade/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.440337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d066f71973b436fb46492fc86196f73, user_id=508b7eb0e9934216932bb1a1fc68aade, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:49.442991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:49.454042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1334/2683] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:49 2022] PUT /identity/v3/projects/2d066f71973b436fb46492fc86196f73/users/508b7eb0e9934216932bb1a1fc68aade/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:49.464799 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.465202 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.465729 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.497431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] MFA Rules not processed for user `508b7eb0e9934216932bb1a1fc68aade`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:49.498750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:49.581001 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1350/2684] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:49 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:49.592086 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:49.594011 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:49.623998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:49.657074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a82f9b-d422-4844-bc83-4f4263c68835 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:49.662369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788e8f3f-0df5-4a2a-a116-bf770ffab7f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:49.662989 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8f3f-0df5-4a2a-a116-bf770ffab7f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:49.663259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8f3f-0df5-4a2a-a116-bf770ffab7f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:49.663532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788e8f3f-0df5-4a2a-a116-bf770ffab7f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:49.674206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e8f3f-0df5-4a2a-a116-bf770ffab7f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:49.675840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788e8f3f-0df5-4a2a-a116-bf770ffab7f4 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:49.686529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1335/2685] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:49 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:50.865856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:50.945183 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23c5df1-193d-465b-8bd5-0183049ad675 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:50.950207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:50.950863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:50.951146 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:50.951420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] PATH_INFO: `/v3/users/3e9bf589176c4889b1573aef4fa7861c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:50.952602 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e9bf589176c4889b1573aef4fa7861c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:50.954609 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:51.143805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e9bf589176c4889b1573aef4fa7861c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:51.144910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e9bf589176c4889b1573aef4fa7861c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:51.146449 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e9bf589176c4889b1573aef4fa7861c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:51.156214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e9bf589176c4889b1573aef4fa7861c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:51.171767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e9bf589176c4889b1573aef4fa7861c'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:51.248230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1351/2686] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:50 2022] DELETE /identity/v3/users/3e9bf589176c4889b1573aef4fa7861c => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:51.254698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:51.254698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:51.254698 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:51.273956 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:51.279704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:51.346332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1336/2687] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:51.352993 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:51.354482 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:51.360847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:51.427594 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82bd868-7362-435a-b1cf-cb08eb54c9dd admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:51.431320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:51.431931 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:51.432247 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:51.432522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:51.442548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:51.443971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:51.453566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1352/2688] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:51.968311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:51.974025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:52.018380 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bce324-b2cc-4ec3-ac77-9d13513855d3 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:52.023370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26387072-744d-4198-940c-29382d18711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:52.024020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26387072-744d-4198-940c-29382d18711c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:52.024297 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26387072-744d-4198-940c-29382d18711c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:52.024616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26387072-744d-4198-940c-29382d18711c admin admin] PATH_INFO: `/v3/projects/5b786d3c743d49088b5ac3d72413d3fa` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:52.025945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26387072-744d-4198-940c-29382d18711c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b786d3c743d49088b5ac3d72413d3fa)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:52.027845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26387072-744d-4198-940c-29382d18711c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:52.172499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-26387072-744d-4198-940c-29382d18711c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b786d3c743d49088b5ac3d72413d3fa', 'request_id': 'req-26387072-744d-4198-940c-29382d18711c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.180253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-26387072-744d-4198-940c-29382d18711c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b786d3c743d49088b5ac3d72413d3fa', 'request_id': 'req-26387072-744d-4198-940c-29382d18711c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.191174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-26387072-744d-4198-940c-29382d18711c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b786d3c743d49088b5ac3d72413d3fa', 'request_id': 'req-26387072-744d-4198-940c-29382d18711c'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.327737 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-26387072-744d-4198-940c-29382d18711c admin admin] The token cache is being invalidate because project 5b786d3c743d49088b5ac3d72413d3fa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:52.328589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-26387072-744d-4198-940c-29382d18711c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.329198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-26387072-744d-4198-940c-29382d18711c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.335093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1337/2689] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:51 2022] DELETE /identity/v3/projects/5b786d3c743d49088b5ac3d72413d3fa => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:52.336101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:52.347645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29ba1cc-8103-4c0b-a57d-cc6ded3b2124 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:52.437293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:52.437935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:52.438244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:52.438691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] PATH_INFO: `/v3/users/508b7eb0e9934216932bb1a1fc68aade` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:52.440051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] RBAC: Authorizing `identity:delete_user(user_id=508b7eb0e9934216932bb1a1fc68aade)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:52.449383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:52.706730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '508b7eb0e9934216932bb1a1fc68aade'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.710596 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '508b7eb0e9934216932bb1a1fc68aade'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.711553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '508b7eb0e9934216932bb1a1fc68aade'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.722289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '508b7eb0e9934216932bb1a1fc68aade'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.748428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '508b7eb0e9934216932bb1a1fc68aade'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:52.849762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1353/2690] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:52 2022] DELETE /identity/v3/users/508b7eb0e9934216932bb1a1fc68aade => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:52.918220 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26387072-744d-4198-940c-29382d18711c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:52.919976 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-26387072-744d-4198-940c-29382d18711c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:52.982290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:53.002578 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26387072-744d-4198-940c-29382d18711c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.060664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.063417 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.071474 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.071971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.072252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.072521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.074932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.092730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.094622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.105935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:53.107246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1338/2691] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:52 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:53.153954 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545dea61-38ae-464c-b9da-d277b6a86342 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.160042 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.160814 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.161199 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.161471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] PATH_INFO: `/v3/users/ebbf49534a8f41109f51637307270016` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.162621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebbf49534a8f41109f51637307270016)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.164626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.302065 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.303900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.315927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebbf49534a8f41109f51637307270016'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.316326 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebbf49534a8f41109f51637307270016'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.317371 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebbf49534a8f41109f51637307270016'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.320035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a0959-ad0a-4a2d-b2bb-9170612aa44a service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.324868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.325182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebbf49534a8f41109f51637307270016'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.325618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.325966 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.326302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] PATH_INFO: `/v3/projects/2d066f71973b436fb46492fc86196f73` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.327452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d066f71973b436fb46492fc86196f73)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.329690 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.340510 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-1ccd6bc5-d5c1-4bab-a7f7-88581fe16752 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebbf49534a8f41109f51637307270016'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.407866 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1354/2692] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:53 2022] DELETE /identity/v3/users/ebbf49534a8f41109f51637307270016 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:53.420598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d066f71973b436fb46492fc86196f73', 'request_id': 'req-0bb56e48-6f0d-470a-b5c6-3787626adcd9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.426114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d066f71973b436fb46492fc86196f73', 'request_id': 'req-0bb56e48-6f0d-470a-b5c6-3787626adcd9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.432413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d066f71973b436fb46492fc86196f73', 'request_id': 'req-0bb56e48-6f0d-470a-b5c6-3787626adcd9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.498229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] The token cache is being invalidate because project 2d066f71973b436fb46492fc86196f73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:53.498523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.499186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-0bb56e48-6f0d-470a-b5c6-3787626adcd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.500735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1339/2693] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:53 2022] DELETE /identity/v3/projects/2d066f71973b436fb46492fc86196f73 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:53.507532 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.507919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.508313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.529608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:53.530818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.598259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1355/2694] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:08:53.606458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.606744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.607017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.628063 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:53.629351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.637411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1340/2695] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:53.641755 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.651995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76953758-ab80-4198-b698-ed803ce4ff0a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.659669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.659669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.659669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.659669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.660018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:08:53.660018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.661032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.672738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1356/2696] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:08:53.677832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.681864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.683836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.687883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d431c0-bb90-4f1e-8139-9120654bcb06 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.694229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.695274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.695557 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.695822 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.696614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.698518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a786203d-ee99-4a38-8133-d9bc02cc87e8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.698673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.704622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.705462 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.705842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.706218 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] PATH_INFO: `/v3/projects/f4f9539894f147e0912a72a1855d31bc` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.707610 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4f9539894f147e0912a72a1855d31bc)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.709901 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.763359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1341/2697] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:53 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:53.764365 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.765914 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.806982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:53.808552 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:53.825221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833c12c8-127d-4bbd-bf52-f535bba76c64 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.825427 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4f9539894f147e0912a72a1855d31bc', 'request_id': 'req-ef220a6d-f210-449b-b8ba-8208a6ebafc5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.831835 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4f9539894f147e0912a72a1855d31bc', 'request_id': 'req-ef220a6d-f210-449b-b8ba-8208a6ebafc5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.874994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.875516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.875791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.876057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.877894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.893515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4f9539894f147e0912a72a1855d31bc', 'request_id': 'req-ef220a6d-f210-449b-b8ba-8208a6ebafc5'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.905794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] The token cache is being invalidate because project f4f9539894f147e0912a72a1855d31bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:53.906080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.906763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:53.909177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1357/2698] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:53 2022] DELETE /identity/v3/projects/f4f9539894f147e0912a72a1855d31bc => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:53.909672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:53.911439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.946620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.948075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8594e0a2-9de5-46f2-afa6-36ac5e2df9ce service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:53.959072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1342/2699] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:53.960332 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780ea4b-7620-42c8-9bb8-e57f63873349 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.960615 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780ea4b-7620-42c8-9bb8-e57f63873349 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.960886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0780ea4b-7620-42c8-9bb8-e57f63873349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.985231 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef220a6d-f210-449b-b8ba-8208a6ebafc5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:53.986955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0780ea4b-7620-42c8-9bb8-e57f63873349 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:53.988302 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0780ea4b-7620-42c8-9bb8-e57f63873349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:53.990805 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:53.991420 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:53.992080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:53.992080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:53.992935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:53.994858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.005606 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1343/2700] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:54.007179 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.007498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.007776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.029526 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:54.032001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:54.041119 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1344/2701] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:08:54.045199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.054572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd7740d-fefc-454f-a56b-488268c07302 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.059251 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.059847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.060123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.060389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.061051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:08:54.061362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.062930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.075645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1345/2702] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 23:08:54.079728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.090196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69faa169-4b4e-4c37-b60c-d6dc5051b5da admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.094931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.095566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.095859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.096136 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.096767 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.098703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.151378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1358/2703] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:53 2022] POST /identity/v3/users => generated 288 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:08:54.155345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.165726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b0debe-5de8-4d1f-b7b8-8d6a4a2fb41d admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.171547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.173740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.173740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.173740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.173740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.174678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1346/2704] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/projects => generated 391 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:08:54.174960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.179058 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.187213 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1359/2705] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:08:54.193983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.204393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafca932-72f3-48c0-a7cc-32b7cc61e16a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.247008 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:54.262482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4284cd95-2f34-468a-9935-7c9f0fd1222a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.270222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.270845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.271129 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.271426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.272037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.274007 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.279976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.280616 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.280920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.281228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.281880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.283823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.294885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1360/2706] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:08:54.298976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.309671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857e2aae-09e7-44cf-b252-b67214bed528 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.315677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7082cb07-ba0e-452b-a1d0-ca043ee4e3e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.316797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082cb07-ba0e-452b-a1d0-ca043ee4e3e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.317252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082cb07-ba0e-452b-a1d0-ca043ee4e3e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.317508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7082cb07-ba0e-452b-a1d0-ca043ee4e3e9 admin admin] PATH_INFO: `/v3/projects/15c5ea7298114947802f96ad13470298/users/0450319f11114c0780bd05afdb8ced63/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.336775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082cb07-ba0e-452b-a1d0-ca043ee4e3e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c5ea7298114947802f96ad13470298, user_id=0450319f11114c0780bd05afdb8ced63, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.340084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7082cb07-ba0e-452b-a1d0-ca043ee4e3e9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.363511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1347/2707] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/users => generated 296 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:08:54.367536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.379697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8f11f-a2cd-4a4f-a9d3-b752d7f9438b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.385315 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.385941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.386222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.386496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.387148 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.388778 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.399485 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1348/2708] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:54.403569 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1361/2709] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:54 2022] PUT /identity/v3/projects/15c5ea7298114947802f96ad13470298/users/0450319f11114c0780bd05afdb8ced63/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:54.404060 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.414477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec670888-9ee5-4616-8cbb-355e03edadeb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.415660 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.416305 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.416877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.428979 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:54.454715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] MFA Rules not processed for user `0450319f11114c0780bd05afdb8ced63`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:54.456201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:54.506411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.506736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.507091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.507319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.508006 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.509848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.524569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1349/2710] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:54.526367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.536621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1362/2711] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:54.540915 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1962ae-0a2f-410a-a490-dca97451fecf admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.546703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da4255-4906-4622-829d-4ae7254d6083 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.549254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da4255-4906-4622-829d-4ae7254d6083 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.549254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da4255-4906-4622-829d-4ae7254d6083 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.549254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da4255-4906-4622-829d-4ae7254d6083 admin admin] PATH_INFO: `/v3/projects/ef2bb1c0b9304ab09581b1afeb7b6295/users/f3425bc8e0114f69b1b66450562b3eb8/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.551823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.553389 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:54.572835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da4255-4906-4622-829d-4ae7254d6083 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef2bb1c0b9304ab09581b1afeb7b6295, user_id=f3425bc8e0114f69b1b66450562b3eb8, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.575626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8da4255-4906-4622-829d-4ae7254d6083 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.612782 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295927a-ba30-4a9a-a3e1-eaeddd96c24d None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.616802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.617460 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.617745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.620981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.627930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.629676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.639177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1363/2712] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:08:54.651966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.652517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:54.656928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1350/2713] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:54 2022] PUT /identity/v3/projects/ef2bb1c0b9304ab09581b1afeb7b6295/users/f3425bc8e0114f69b1b66450562b3eb8/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:54.667694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbf16f0-60f8-4f88-b0a0-580c814e210c service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.667928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.668488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.668961 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.701649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] MFA Rules not processed for user `f3425bc8e0114f69b1b66450562b3eb8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:54.703319 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:54.733826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.734910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.735382 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.736030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] PATH_INFO: `/v3/users/adc12894a7c94ae1b06340353fd99654` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.738006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] RBAC: Authorizing `identity:delete_user(user_id=adc12894a7c94ae1b06340353fd99654)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.739928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.780964 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1351/2714] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:08:54.785883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.796447 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa21ae2-fe7f-4765-83ce-d93b2c325ae1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.801733 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.802335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.802614 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.802883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.803516 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.805488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:54.906593 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1352/2715] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/projects => generated 391 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:08:54.911452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:54.924782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:54.939092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:54.947858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'adc12894a7c94ae1b06340353fd99654'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:54.948538 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'adc12894a7c94ae1b06340353fd99654'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:54.949250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'adc12894a7c94ae1b06340353fd99654'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:54.956593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'adc12894a7c94ae1b06340353fd99654'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:54.969418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'adc12894a7c94ae1b06340353fd99654'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:54.978041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac0ae88-a309-473d-a389-5b031f932b29 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:54.982723 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:54.983329 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:54.983610 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:54.983884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:54.984504 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:54.987140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.053481 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1364/2716] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:54 2022] DELETE /identity/v3/users/adc12894a7c94ae1b06340353fd99654 => generated 0 bytes in 412 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:08:55.111184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1353/2717] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:08:54 2022] POST /identity/v3/users => generated 302 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:08:55.115363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.116881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.186588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd10b73-c5ee-42ac-a7eb-15981274d384 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.191464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.192080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.192356 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.192626 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.193293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.194930 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.207162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1365/2718] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:08:55.211697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.226594 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474d12de-43b7-4068-b3c2-bb832eaa8d39 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.244390 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.245138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.245416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.245688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] PATH_INFO: `/v3/projects/26a14aa6af4c4cbeba9f2410570dbd99/users/c9b4179891f94961ae3fb5e9d664f8c8/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.265438 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a14aa6af4c4cbeba9f2410570dbd99, user_id=c9b4179891f94961ae3fb5e9d664f8c8, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.268735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.353658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.357073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.368747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1354/2719] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:55 2022] PUT /identity/v3/projects/26a14aa6af4c4cbeba9f2410570dbd99/users/c9b4179891f94961ae3fb5e9d664f8c8/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:55.373325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a982e3-8220-4dca-85f6-65e8d6f092e1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.373701 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.444040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.444612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.445120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.445345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] PATH_INFO: `/v3/projects/182b0a6a36fb4c14816d9f94b270686c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.446547 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058bdcaf-5ff1-426e-a7ac-01894de4e8f6 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.446701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=182b0a6a36fb4c14816d9f94b270686c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.448681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.451566 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.452205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.458602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.458602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.458602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.458602 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.466777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1355/2720] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:08:55.472203 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.483256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801baa73-4a49-47e8-851e-ffb285883fda admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.488589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f6a377-794d-4f45-a6fe-39410335dde6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.489313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a377-794d-4f45-a6fe-39410335dde6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.489590 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a377-794d-4f45-a6fe-39410335dde6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.489865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f6a377-794d-4f45-a6fe-39410335dde6 admin admin] PATH_INFO: `/v3/projects/26a14aa6af4c4cbeba9f2410570dbd99/users/c9b4179891f94961ae3fb5e9d664f8c8/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.491902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6a377-794d-4f45-a6fe-39410335dde6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26a14aa6af4c4cbeba9f2410570dbd99, user_id=c9b4179891f94961ae3fb5e9d664f8c8, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.494638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f6a377-794d-4f45-a6fe-39410335dde6 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.506496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1356/2721] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:08:55 2022] PUT /identity/v3/projects/26a14aa6af4c4cbeba9f2410570dbd99/users/c9b4179891f94961ae3fb5e9d664f8c8/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:55.518946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.525188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.525188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.549625 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] MFA Rules not processed for user `c9b4179891f94961ae3fb5e9d664f8c8`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:08:55.550896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.602320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182b0a6a36fb4c14816d9f94b270686c', 'request_id': 'req-b71fd448-c86e-492d-867c-6e8c2e67aaa7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:55.607561 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:55.610384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182b0a6a36fb4c14816d9f94b270686c', 'request_id': 'req-b71fd448-c86e-492d-867c-6e8c2e67aaa7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:55.616700 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:55.637846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1357/2722] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:08:55 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:08:55.650367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.652200 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.709021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07033b0-2426-4f1b-bacf-d48015ff412f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.721844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182b0a6a36fb4c14816d9f94b270686c', 'request_id': 'req-b71fd448-c86e-492d-867c-6e8c2e67aaa7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:55.734323 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] The token cache is being invalidate because project 182b0a6a36fb4c14816d9f94b270686c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:08:55.734586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:55.735505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:08:55.737300 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1366/2723] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:08:55 2022] DELETE /identity/v3/projects/182b0a6a36fb4c14816d9f94b270686c => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:08:55.762184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.764053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.764053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.764053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.765500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.840343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.841848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:55.853138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1358/2724] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:55 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:08:55.906085 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:55.907524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.920358 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71fd448-c86e-492d-867c-6e8c2e67aaa7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:55.924638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:55.925279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:55.925554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:55.925818 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:55.927586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:55.998091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:55.998091 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab80bfb-87bd-4237-ac6c-e0164fc05d2b service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:08:56.007944 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1367/2725] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:08:55 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:08:59.860031 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:08:59.861708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:08:59.919155 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=100 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:08:59.932140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2543853-7ccc-4234-b72b-fa6b74df2f79 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:08:59.936720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:08:59.937472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:08:59.937747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:08:59.938017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] PATH_INFO: `/v3/users/0450319f11114c0780bd05afdb8ced63` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:08:59.939237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0450319f11114c0780bd05afdb8ced63)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:08:59.941277 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:00.113789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0450319f11114c0780bd05afdb8ced63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:00.114576 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0450319f11114c0780bd05afdb8ced63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:00.128030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0450319f11114c0780bd05afdb8ced63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:00.128879 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0450319f11114c0780bd05afdb8ced63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:00.136572 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0450319f11114c0780bd05afdb8ced63'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:00.231280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1359/2726] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:08:59 2022] DELETE /identity/v3/users/0450319f11114c0780bd05afdb8ced63 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:00.235959 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:00.236501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:00.236501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:00.256808 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:00.258095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:00.322412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1368/2727] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:00.333422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:00.334843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:00.392823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c652cf-a0a2-4ae0-9831-8920cf2b04c0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:00.396492 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf480ef-0b1a-4590-854d-7da46981de53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:00.397118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf480ef-0b1a-4590-854d-7da46981de53 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:00.397398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf480ef-0b1a-4590-854d-7da46981de53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:00.397664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf480ef-0b1a-4590-854d-7da46981de53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:00.407046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf480ef-0b1a-4590-854d-7da46981de53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:00.408775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf480ef-0b1a-4590-854d-7da46981de53 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:00.418641 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1360/2728] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:09:00.930525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:00.932652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:00.944962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ddfef-b215-407c-8908-377d8b527e86 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:00.949863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:00.955258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:00.955258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:00.955258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] PATH_INFO: `/v3/projects/15c5ea7298114947802f96ad13470298` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:00.955258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c5ea7298114947802f96ad13470298)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:00.955258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.154628 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c5ea7298114947802f96ad13470298', 'request_id': 'req-a4227a7a-8446-400b-be2f-8b9e940cbacb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:01.161738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c5ea7298114947802f96ad13470298', 'request_id': 'req-a4227a7a-8446-400b-be2f-8b9e940cbacb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:01.237058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c5ea7298114947802f96ad13470298', 'request_id': 'req-a4227a7a-8446-400b-be2f-8b9e940cbacb'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:01.250406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] The token cache is being invalidate because project 15c5ea7298114947802f96ad13470298 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:01.250802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:01.251475 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a4227a7a-8446-400b-be2f-8b9e940cbacb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:01.253142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1369/2729] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:00 2022] DELETE /identity/v3/projects/15c5ea7298114947802f96ad13470298 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:01.258162 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.258684 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.258894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.282127 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:01.283384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:01.345916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1361/2730] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:01.354473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.354764 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.357138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.363927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:01.375504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:01.376712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:01.385103 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1370/2731] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:01.389335 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:01.399916 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b6dd2-b0d0-4904-ab62-a20feb15a9c4 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:01.405638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:01.406256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.408846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.408846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.408846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:09:01.408846 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:01.410350 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.419739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:01.425206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1362/2732] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:09:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:09:01.427935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:01.437252 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15158510-fd9e-4537-a470-4b1c19e33b72 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:01.442142 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:01.442740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.443015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.443283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.443929 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:01.445849 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.580293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1371/2733] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:01 2022] POST /identity/v3/projects => generated 367 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:01.585305 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:01.595565 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf48f29-8440-4a9b-b082-10e57db32a9f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:01.653102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:01.653666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.653975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.654247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.654874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:01.656910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.766371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1363/2734] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:01 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:01.767714 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:01.779004 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a821966-9562-4f17-bafe-7dd0c73bf7d5 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:01.784038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:01.784641 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.784921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.785244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.785895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:01.787531 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.797662 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1372/2735] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:01.801943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:01.812159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87e1878-d5d9-409f-9286-ea6360cf299c admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:01.817498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49610a34-5074-44cd-8cc2-8c0159a8b3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:01.818100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49610a34-5074-44cd-8cc2-8c0159a8b3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.818377 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49610a34-5074-44cd-8cc2-8c0159a8b3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.818679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49610a34-5074-44cd-8cc2-8c0159a8b3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.819322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49610a34-5074-44cd-8cc2-8c0159a8b3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:01.820896 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49610a34-5074-44cd-8cc2-8c0159a8b3f9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.832460 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1364/2736] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:01.836536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:01.846548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb37637c-49e5-42ba-8d75-7616b7c892b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:01.851584 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:01.852238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.852514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.852786 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] PATH_INFO: `/v3/projects/343c907da4ea423e8f1ef6c13bd576e8/users/4db176d0edeb4d2e8c438a0e675dbac1/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:01.867797 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=343c907da4ea423e8f1ef6c13bd576e8, user_id=4db176d0edeb4d2e8c438a0e675dbac1, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:01.870436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:01.968110 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1373/2737] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:01 2022] PUT /identity/v3/projects/343c907da4ea423e8f1ef6c13bd576e8/users/4db176d0edeb4d2e8c438a0e675dbac1/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:01.978782 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:01.979067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:01.979344 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:02.009109 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] MFA Rules not processed for user `4db176d0edeb4d2e8c438a0e675dbac1`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:02.010351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:02.075682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1365/2738] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:02.087658 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:02.089377 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:02.149047 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f3165ba-1657-49f6-a0ee-7da3575a68d7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:02.153171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e15616b-a2d8-468f-a867-7a8ecf19bbb7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:02.153785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15616b-a2d8-468f-a867-7a8ecf19bbb7 service glance] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:02.154063 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15616b-a2d8-468f-a867-7a8ecf19bbb7 service glance] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:02.154337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e15616b-a2d8-468f-a867-7a8ecf19bbb7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:02.163587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15616b-a2d8-468f-a867-7a8ecf19bbb7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:02.175228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e15616b-a2d8-468f-a867-7a8ecf19bbb7 service glance] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:02.176374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1374/2739] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:02 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:02.186630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:02.188052 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:02.258398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d55d6c-98a4-4b2a-9357-411d706eb3f7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:02.263301 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:02.263936 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:02.264221 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:02.264495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] PATH_INFO: `/v3/users/f3425bc8e0114f69b1b66450562b3eb8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:02.265749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3425bc8e0114f69b1b66450562b3eb8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:02.267669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:02.414945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:02.424352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3425bc8e0114f69b1b66450562b3eb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:02.425670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3425bc8e0114f69b1b66450562b3eb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:02.438981 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3425bc8e0114f69b1b66450562b3eb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:02.444596 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3425bc8e0114f69b1b66450562b3eb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:02.447528 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3425bc8e0114f69b1b66450562b3eb8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:02.510970 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1366/2740] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:02 2022] DELETE /identity/v3/users/f3425bc8e0114f69b1b66450562b3eb8 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:02.515514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:02.515980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:02.516093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:02.536307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:02.537651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:02.600022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1375/2741] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:02.610697 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:02.612169 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:02.672500 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80243f6-f5cf-40d3-9f0a-bbb189fa2986 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:02.677067 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:02.677679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:02.677962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:02.678231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:02.687406 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:02.688836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:02.698699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1367/2742] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:03.147048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:03.148637 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:03.161141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2f3b68-1443-4522-9e01-871e0761ce5e None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:03.165789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:03.166428 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:03.166701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:03.166972 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] PATH_INFO: `/v3/projects/ef2bb1c0b9304ab09581b1afeb7b6295` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:03.168095 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef2bb1c0b9304ab09581b1afeb7b6295)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:03.170037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:03.282055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:03.288984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef2bb1c0b9304ab09581b1afeb7b6295', 'request_id': 'req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.294265 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef2bb1c0b9304ab09581b1afeb7b6295', 'request_id': 'req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.373201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef2bb1c0b9304ab09581b1afeb7b6295', 'request_id': 'req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.385443 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] The token cache is being invalidate because project ef2bb1c0b9304ab09581b1afeb7b6295 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:03.385733 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.386378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.387875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1376/2743] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:03 2022] DELETE /identity/v3/projects/ef2bb1c0b9304ab09581b1afeb7b6295 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:03.391921 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:03.404110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f519ad0-5fa7-4076-bc42-a6344d3c614d service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:03.472630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:03.473340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:03.473621 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:03.473898 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] PATH_INFO: `/v3/users/c9b4179891f94961ae3fb5e9d664f8c8` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:03.475048 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9b4179891f94961ae3fb5e9d664f8c8)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:03.477165 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:03.667181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:DELETE=15,keystone:SELECT=84,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:03.681123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9b4179891f94961ae3fb5e9d664f8c8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.681799 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9b4179891f94961ae3fb5e9d664f8c8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.695720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9b4179891f94961ae3fb5e9d664f8c8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.696443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9b4179891f94961ae3fb5e9d664f8c8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.705443 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-db7f383c-99dd-48d3-b3d9-e9607c17dc19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9b4179891f94961ae3fb5e9d664f8c8'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:03.813725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1368/2744] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:03 2022] DELETE /identity/v3/users/c9b4179891f94961ae3fb5e9d664f8c8 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:04.369897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:04.371426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:04.386871 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70e21b0-fa35-4360-b40e-5db7fc6ae4a6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:04.457315 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:04.457965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:04.458244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:04.458523 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] PATH_INFO: `/v3/projects/26a14aa6af4c4cbeba9f2410570dbd99` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:04.459841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=26a14aa6af4c4cbeba9f2410570dbd99)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:04.461909 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:04.589536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26a14aa6af4c4cbeba9f2410570dbd99', 'request_id': 'req-a973727d-09e7-4c3c-8115-be204c510b9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:04.595435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26a14aa6af4c4cbeba9f2410570dbd99', 'request_id': 'req-a973727d-09e7-4c3c-8115-be204c510b9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:04.665444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26a14aa6af4c4cbeba9f2410570dbd99', 'request_id': 'req-a973727d-09e7-4c3c-8115-be204c510b9d'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:04.678294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] The token cache is being invalidate because project 26a14aa6af4c4cbeba9f2410570dbd99 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:04.678841 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:04.679566 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a973727d-09e7-4c3c-8115-be204c510b9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:04.681535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1377/2745] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:04 2022] DELETE /identity/v3/projects/26a14aa6af4c4cbeba9f2410570dbd99 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:04.686358 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:04.686941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:04.687038 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:04.708464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:04.724835 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:04.770923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1369/2746] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:04.778751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:04.779038 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:04.779318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:04.813677 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:04.814908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:04.834077 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1378/2747] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:04.838599 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:04.849763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1371c4-d0cb-4f1d-9b66-b94197514109 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:04.854484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:04.855122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:04.855331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:04.855601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:04.856250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:09:04.856551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:04.858202 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:04.874739 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1370/2748] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:09:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:09:04.880078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:04.900963 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ee198f-b802-4646-84c2-6064cf9b5980 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:04.906282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:04.906899 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:04.907182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:04.907455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:04.908089 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:04.910700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.014657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:05.024361 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1379/2749] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:04 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:05.026550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.048958 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd12a96-c3af-4083-9e32-c743fbb343fa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.116465 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.117413 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.117844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.118118 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.118790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.120652 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.241074 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1371/2750] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:05 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:05.242254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.254827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07c4e19-6a03-4dd9-8a13-c65a0c119551 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.260179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.260789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.261121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.261396 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.262041 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.263645 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.276477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1380/2751] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:09:05.279521 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.290057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa3e04a-b627-45a5-8cc5-d8833254a415 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.294972 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f56e646-f471-4c46-b9cc-692eeae64dd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.298514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e646-f471-4c46-b9cc-692eeae64dd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.298514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e646-f471-4c46-b9cc-692eeae64dd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.298514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f56e646-f471-4c46-b9cc-692eeae64dd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.298514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56e646-f471-4c46-b9cc-692eeae64dd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.298514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f56e646-f471-4c46-b9cc-692eeae64dd4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.308978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1372/2752] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:05.313391 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.324812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37460897-5639-4a4b-b591-f3b5bbe1b875 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.329555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.330207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.330489 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.331401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] PATH_INFO: `/v3/projects/c073fa1f7cc6478a831f17e0ed22bea5/users/52b742751f7d43b9ad65e82e96f2ef0d/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.346894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c073fa1f7cc6478a831f17e0ed22bea5, user_id=52b742751f7d43b9ad65e82e96f2ef0d, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.349587 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.408507 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1381/2753] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:05 2022] PUT /identity/v3/projects/c073fa1f7cc6478a831f17e0ed22bea5/users/52b742751f7d43b9ad65e82e96f2ef0d/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:05.423347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.423729 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.424017 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.438102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:05.453648 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] MFA Rules not processed for user `52b742751f7d43b9ad65e82e96f2ef0d`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:05.454900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:05.524759 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1373/2754] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:05.529920 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.598955 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446b5c71-cea0-41c0-9827-f39e911fb9c9 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.604311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.604921 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.605279 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.605530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.606169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.608090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.730597 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1382/2755] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:05 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:05.734666 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.746317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8548029c-cc5a-48a1-973f-3fc36f0b5904 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.807704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.808310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.808707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.808991 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.809890 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.812113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.934579 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1374/2756] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:05 2022] POST /identity/v3/users => generated 297 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:09:05.940690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.951834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7075e566-3745-4e47-8526-c5b4674fc928 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.957511 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.958165 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.958527 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.958802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:05.959471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:05.961125 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:05.973593 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1383/2757] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:05.978539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:05.989235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17aaeed-1633-48ff-b7e7-3fab42368a8b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:05.993937 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:05.994598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:05.994880 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:05.995152 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] PATH_INFO: `/v3/projects/bad65efc01ea4ecf858bcae3e92841d2/users/c1178d756b6e42cb9f0db424ab0567e3/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:06.012300 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad65efc01ea4ecf858bcae3e92841d2, user_id=c1178d756b6e42cb9f0db424ab0567e3, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:06.014992 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:06.089573 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1375/2758] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:05 2022] PUT /identity/v3/projects/bad65efc01ea4ecf858bcae3e92841d2/users/c1178d756b6e42cb9f0db424ab0567e3/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:09:06.093608 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:06.104957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb221a6-06e1-48bf-979b-25c5aafe71eb admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:06.168293 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7906a9-47ce-4454-b066-c41a9eac3315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:06.168912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7906a9-47ce-4454-b066-c41a9eac3315 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:06.169244 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7906a9-47ce-4454-b066-c41a9eac3315 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:06.169542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7906a9-47ce-4454-b066-c41a9eac3315 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:06.170201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7906a9-47ce-4454-b066-c41a9eac3315 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:06.171873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7906a9-47ce-4454-b066-c41a9eac3315 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:06.188352 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1384/2759] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:06.189933 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:06.201607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4eb2f9-5a74-4b71-a92c-c9a76a749866 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:06.210175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:06.210993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:06.211309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:06.211604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] PATH_INFO: `/v3/projects/bad65efc01ea4ecf858bcae3e92841d2/users/c1178d756b6e42cb9f0db424ab0567e3/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:06.213622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bad65efc01ea4ecf858bcae3e92841d2, user_id=c1178d756b6e42cb9f0db424ab0567e3, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:06.216303 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:06.228676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1376/2760] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:06 2022] PUT /identity/v3/projects/bad65efc01ea4ecf858bcae3e92841d2/users/c1178d756b6e42cb9f0db424ab0567e3/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:06.240528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:06.240817 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:06.241140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:06.273516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] MFA Rules not processed for user `c1178d756b6e42cb9f0db424ab0567e3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:06.274775 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:06.299620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:06.346556 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1385/2761] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:06 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:06.357788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:06.359494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:06.415895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:06.420969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39a2d4-456d-4bb3-aef1-e858c3b3cebc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:06.424746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:06.425385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:06.425713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:06.425985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:06.435556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:06.438267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:06.449123 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1377/2762] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:06 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:06.555624 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:06.566098 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f18cd3e-0d63-4ae5-8e32-38bf75def2d1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:06.569736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2092ff8-4e10-42d8-91a7-0c7d714dc1a8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:06.570342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092ff8-4e10-42d8-91a7-0c7d714dc1a8 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:06.570618 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092ff8-4e10-42d8-91a7-0c7d714dc1a8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:06.570884 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2092ff8-4e10-42d8-91a7-0c7d714dc1a8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:06.631501 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092ff8-4e10-42d8-91a7-0c7d714dc1a8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:06.632859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2092ff8-4e10-42d8-91a7-0c7d714dc1a8 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:06.642015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1386/2763] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:10.906001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:10.974617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e0ce9-8178-4a80-a1b1-8939daea1cb4 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:10.979163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:10.979974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:10.980103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:10.980318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] PATH_INFO: `/v3/users/52b742751f7d43b9ad65e82e96f2ef0d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:10.981458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b742751f7d43b9ad65e82e96f2ef0d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:10.983362 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:11.305730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b742751f7d43b9ad65e82e96f2ef0d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:11.306494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b742751f7d43b9ad65e82e96f2ef0d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:11.320407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b742751f7d43b9ad65e82e96f2ef0d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:11.321254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b742751f7d43b9ad65e82e96f2ef0d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:11.333497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b742751f7d43b9ad65e82e96f2ef0d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:11.426147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1378/2764] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:10 2022] DELETE /identity/v3/users/52b742751f7d43b9ad65e82e96f2ef0d => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:11.431138 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:11.432167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:11.432453 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:11.468882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:11.470190 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:11.571806 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1387/2765] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:11.583744 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:11.585258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:11.652093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e5f3e4-51f5-4fdd-8a52-70412dbcdbbe admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:11.656330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:11.656953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:11.657262 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:11.657530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:11.667506 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:11.669331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:11.684558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1379/2766] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:09:12.135893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:12.137627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:12.152333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb4f8d3-aa63-4342-b958-1428c4422e1b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:12.157395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:12.158211 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:12.159595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:12.159595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] PATH_INFO: `/v3/projects/c073fa1f7cc6478a831f17e0ed22bea5` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:12.160691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c073fa1f7cc6478a831f17e0ed22bea5)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:12.163040 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:12.277497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c073fa1f7cc6478a831f17e0ed22bea5', 'request_id': 'req-f47130cf-5005-43d8-9a2a-9583cb46b2c6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.281378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:12.283436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c073fa1f7cc6478a831f17e0ed22bea5', 'request_id': 'req-f47130cf-5005-43d8-9a2a-9583cb46b2c6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.357238 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c073fa1f7cc6478a831f17e0ed22bea5', 'request_id': 'req-f47130cf-5005-43d8-9a2a-9583cb46b2c6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.369579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] The token cache is being invalidate because project c073fa1f7cc6478a831f17e0ed22bea5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:12.369882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.370559 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.372092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1388/2767] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:12 2022] DELETE /identity/v3/projects/c073fa1f7cc6478a831f17e0ed22bea5 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:12.376131 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:12.386894 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bf8426e-e3fc-48ee-bbed-4be582b80022 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:12.397775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:12.451393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:12.452035 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:12.452314 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:12.452581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] PATH_INFO: `/v3/users/c1178d756b6e42cb9f0db424ab0567e3` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:12.453741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c1178d756b6e42cb9f0db424ab0567e3)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:12.455664 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:12.603286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c1178d756b6e42cb9f0db424ab0567e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.603948 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c1178d756b6e42cb9f0db424ab0567e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.617369 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c1178d756b6e42cb9f0db424ab0567e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.618113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c1178d756b6e42cb9f0db424ab0567e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.625070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-15f5921c-f773-41a2-a51d-893e01c1d330 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c1178d756b6e42cb9f0db424ab0567e3'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:12.744488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1380/2768] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:12 2022] DELETE /identity/v3/users/c1178d756b6e42cb9f0db424ab0567e3 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:13.225185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:13.227270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:13.246698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47130cf-5005-43d8-9a2a-9583cb46b2c6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:13.321476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:13.322120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:13.322394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:13.322664 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] PATH_INFO: `/v3/projects/bad65efc01ea4ecf858bcae3e92841d2` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:13.323791 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bad65efc01ea4ecf858bcae3e92841d2)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:13.325724 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:13.449153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bad65efc01ea4ecf858bcae3e92841d2', 'request_id': 'req-414b46c1-fc38-490d-a9d2-a242b625f5c4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:13.454757 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bad65efc01ea4ecf858bcae3e92841d2', 'request_id': 'req-414b46c1-fc38-490d-a9d2-a242b625f5c4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:13.534270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bad65efc01ea4ecf858bcae3e92841d2', 'request_id': 'req-414b46c1-fc38-490d-a9d2-a242b625f5c4'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:13.545873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] The token cache is being invalidate because project bad65efc01ea4ecf858bcae3e92841d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:13.546537 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:13.546895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-414b46c1-fc38-490d-a9d2-a242b625f5c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:13.548769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1389/2769] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:13 2022] DELETE /identity/v3/projects/bad65efc01ea4ecf858bcae3e92841d2 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:13.554899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:13.555549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:13.555549 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:13.577452 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:13.579096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:13.658458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1381/2770] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:13.667014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:13.667295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:13.667571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:13.689855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:13.691633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:13.700196 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1390/2771] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:13 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:13.704804 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:13.716041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62b35a4-b039-4560-a824-8adf10787e29 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:13.720681 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:13.721323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:13.721619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:13.722005 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:13.722928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:09:13.723237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:13.725246 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:13.738531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1382/2772] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:09:13.742933 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:13.752418 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e76c81e-6c6b-4d43-b62f-2f3e6cb8e9f1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:13.757728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:13.758359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:13.758633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:13.758900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:13.759524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:13.761451 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:13.871003 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1391/2773] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:13 2022] POST /identity/v3/projects => generated 381 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:13.872045 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:13.884366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0d291-c67b-4dc5-875f-9aa78b43ab1b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:13.902327 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:13.943927 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:13.944541 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:13.944821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:13.945126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:13.945775 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:13.947699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.037930 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1383/2774] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:13 2022] POST /identity/v3/users => generated 291 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:09:14.042222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.053450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00527f-ef23-459f-8fe7-92201d3e7bb2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.058171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.058813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.059092 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.059363 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.060009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.061652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.073681 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1392/2775] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:14.077789 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.090267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5878595d-fcfe-4d38-be13-56e32c21d483 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.095985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44c871a-d349-423e-9fff-44da49f9dce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.096598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c871a-d349-423e-9fff-44da49f9dce9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.096878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c871a-d349-423e-9fff-44da49f9dce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.097181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44c871a-d349-423e-9fff-44da49f9dce9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.097869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c871a-d349-423e-9fff-44da49f9dce9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.099490 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44c871a-d349-423e-9fff-44da49f9dce9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.110790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1384/2776] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:14.115713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.126262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b336b6c-b8d4-4554-8c9a-9fb1da2d175e admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.131873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.132564 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.132878 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.133219 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] PATH_INFO: `/v3/projects/47df3a044050447e9b1afdad3ad665b6/users/e80f89e8260d428589fb839c9004b647/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.150529 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47df3a044050447e9b1afdad3ad665b6, user_id=e80f89e8260d428589fb839c9004b647, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.153578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.238242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1393/2777] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:14 2022] PUT /identity/v3/projects/47df3a044050447e9b1afdad3ad665b6/users/e80f89e8260d428589fb839c9004b647/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:14.257062 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.257359 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.257622 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.285975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] MFA Rules not processed for user `e80f89e8260d428589fb839c9004b647`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:14.287328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:14.354893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1385/2778] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:14 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:14.359413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.369973 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:14.425777 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903acaa-98f2-4626-a904-3aea58913092 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.429149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.429762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.430068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.430359 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.430988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.432926 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.576005 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1394/2779] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:14 2022] POST /identity/v3/projects => generated 381 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:09:14.580296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.591928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394743d6-0eda-4413-ab78-0c7e5027cd0a None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.654935 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.656175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.656461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.656762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.657644 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.660339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.786672 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1386/2780] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:14 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:14.790794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.801960 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e6cc89-a492-4ab9-8f55-d0fa0cd350e8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.806586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.807197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.807473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.807742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.808388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.810022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.821698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1395/2781] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:14.826254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.836901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d33f3aa-d335-449e-8844-a7a27e4275ad admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:14.841601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:14.842254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:14.846655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:14.846655 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] PATH_INFO: `/v3/projects/22d30f2d117749bf96344ebd466bf8c6/users/e8025dd499904f8d8d2dc86166994623/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:14.859860 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d30f2d117749bf96344ebd466bf8c6, user_id=e8025dd499904f8d8d2dc86166994623, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:14.863360 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:14.865658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:14.929364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1387/2782] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:14 2022] PUT /identity/v3/projects/22d30f2d117749bf96344ebd466bf8c6/users/e8025dd499904f8d8d2dc86166994623/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:14.933700 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:14.944877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e66a203-0cff-486f-be4a-709093eddd09 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:15.001504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639d001c-af28-4d3a-9543-4a634891c325 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:15.002112 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d001c-af28-4d3a-9543-4a634891c325 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:15.002390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d001c-af28-4d3a-9543-4a634891c325 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:15.002702 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639d001c-af28-4d3a-9543-4a634891c325 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:15.003488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d001c-af28-4d3a-9543-4a634891c325 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:15.005120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639d001c-af28-4d3a-9543-4a634891c325 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:15.015292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1396/2783] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:15.019719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:15.030519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d999b7-9694-4c8a-8178-04b24be71b91 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:15.036198 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:15.036824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:15.037140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:15.037415 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] PATH_INFO: `/v3/projects/22d30f2d117749bf96344ebd466bf8c6/users/e8025dd499904f8d8d2dc86166994623/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:15.039372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=22d30f2d117749bf96344ebd466bf8c6, user_id=e8025dd499904f8d8d2dc86166994623, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:15.042015 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:15.053730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1388/2784] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:15 2022] PUT /identity/v3/projects/22d30f2d117749bf96344ebd466bf8c6/users/e8025dd499904f8d8d2dc86166994623/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:15.063865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:15.064692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:15.064692 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:15.094530 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] MFA Rules not processed for user `e8025dd499904f8d8d2dc86166994623`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:15.095778 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:15.174473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1397/2785] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:15 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:15.186703 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:15.190678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:15.253512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7a6812-3bc8-433f-a5c9-17afec4007cd admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:15.257310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c20747-24cd-44e7-ac42-8121b01bd087 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:15.257978 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c20747-24cd-44e7-ac42-8121b01bd087 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:15.258265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c20747-24cd-44e7-ac42-8121b01bd087 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:15.258536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c20747-24cd-44e7-ac42-8121b01bd087 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:15.268990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c20747-24cd-44e7-ac42-8121b01bd087 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:15.270484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c20747-24cd-44e7-ac42-8121b01bd087 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:15.281366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1389/2786] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:15 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:15.870716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:15.876114 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:15.934456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8081e629-0eca-429d-a7da-adf81c3fa606 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:15.939037 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:15.939606 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:15.939881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:15.940148 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] PATH_INFO: `/v3/users/e80f89e8260d428589fb839c9004b647` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:15.941370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e80f89e8260d428589fb839c9004b647)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:15.943289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:16.154650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e80f89e8260d428589fb839c9004b647'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:16.155298 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e80f89e8260d428589fb839c9004b647'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:16.155965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e80f89e8260d428589fb839c9004b647'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:16.163433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e80f89e8260d428589fb839c9004b647'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:16.175392 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e80f89e8260d428589fb839c9004b647'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:16.277919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1398/2787] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:15 2022] DELETE /identity/v3/users/e80f89e8260d428589fb839c9004b647 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:16.282707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:16.283184 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:16.283274 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:16.303072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:16.304318 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:16.365398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1390/2788] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:16.376009 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:16.377500 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:16.436336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd858aec-bc18-463f-a9fa-1b55c1ae9bd7 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:16.439964 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:16.440713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:16.441317 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:16.441455 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:16.454586 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:16.457471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:16.468870 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1399/2789] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:16.902306 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:16.904861 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:16.916028 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dd61a4-1533-41cd-9b8a-87e1156d1090 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:16.920801 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:16.921548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:16.921826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:16.922095 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] PATH_INFO: `/v3/projects/47df3a044050447e9b1afdad3ad665b6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:16.923236 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] RBAC: Authorizing `identity:delete_project(project_id=47df3a044050447e9b1afdad3ad665b6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:16.927631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:17.072786 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47df3a044050447e9b1afdad3ad665b6', 'request_id': 'req-38b2105c-9778-4efb-979c-40723224802f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.078103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47df3a044050447e9b1afdad3ad665b6', 'request_id': 'req-38b2105c-9778-4efb-979c-40723224802f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.084440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47df3a044050447e9b1afdad3ad665b6', 'request_id': 'req-38b2105c-9778-4efb-979c-40723224802f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.180653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] The token cache is being invalidate because project 47df3a044050447e9b1afdad3ad665b6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:17.180965 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.181668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.183137 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1391/2790] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:16 2022] DELETE /identity/v3/projects/47df3a044050447e9b1afdad3ad665b6 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:17.187152 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:17.197867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05633e70-a2b2-457c-a0d7-df73693d2d03 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:17.222877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:17.256024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:17.256652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:17.256928 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:17.257229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] PATH_INFO: `/v3/users/e8025dd499904f8d8d2dc86166994623` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:17.258413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8025dd499904f8d8d2dc86166994623)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:17.260402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:17.496740 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8025dd499904f8d8d2dc86166994623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.497408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8025dd499904f8d8d2dc86166994623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.498015 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8025dd499904f8d8d2dc86166994623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.505591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8025dd499904f8d8d2dc86166994623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.517978 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-06d7dbc4-afc6-4a39-83a9-7fb2c6ec9359 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8025dd499904f8d8d2dc86166994623'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:17.620328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1400/2791] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:17 2022] DELETE /identity/v3/users/e8025dd499904f8d8d2dc86166994623 => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:18.078887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.080309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:18.094081 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b2105c-9778-4efb-979c-40723224802f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.099253 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:INSERT=3,keystone:SELECT=90,keystone:DELETE=7 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:18.158435 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.159249 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.160080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.160401 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] PATH_INFO: `/v3/projects/22d30f2d117749bf96344ebd466bf8c6` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.161843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22d30f2d117749bf96344ebd466bf8c6)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:18.164239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:18.278363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22d30f2d117749bf96344ebd466bf8c6', 'request_id': 'req-aadf106e-0e4f-4995-922d-5968aaa88f88'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:18.283769 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22d30f2d117749bf96344ebd466bf8c6', 'request_id': 'req-aadf106e-0e4f-4995-922d-5968aaa88f88'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:18.290519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22d30f2d117749bf96344ebd466bf8c6', 'request_id': 'req-aadf106e-0e4f-4995-922d-5968aaa88f88'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:18.369675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] The token cache is being invalidate because project 22d30f2d117749bf96344ebd466bf8c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:18.369871 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:18.370560 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-aadf106e-0e4f-4995-922d-5968aaa88f88 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:18.372082 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1392/2792] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:18 2022] DELETE /identity/v3/projects/22d30f2d117749bf96344ebd466bf8c6 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:18.377224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcd060-926b-4049-b094-e416724efa1a None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.377784 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcd060-926b-4049-b094-e416724efa1a None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.377784 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fcd060-926b-4049-b094-e416724efa1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.401182 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-29fcd060-926b-4049-b094-e416724efa1a None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:18.402713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-29fcd060-926b-4049-b094-e416724efa1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:18.473276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1401/2793] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:18.481284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.481662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.481943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.504466 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:18.505780 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:18.514088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1393/2794] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:18.518548 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcd060-926b-4049-b094-e416724efa1a None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.529750 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fcd060-926b-4049-b094-e416724efa1a None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.535535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.536169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.536458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.536731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.537436 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:09:18.537743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:18.539413 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:18.552370 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1402/2795] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:09:18.557096 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.567347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70969f9-0270-4146-b8c1-ade5071f359c None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.572257 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.572929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.573240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.573512 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.574159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:18.576071 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:18.690531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1394/2796] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:18 2022] POST /identity/v3/projects => generated 385 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:18.694738 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.704907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f685767-d2e4-4454-9e57-6e6466b39ba0 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.736703 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:18.764623 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.765301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.765612 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.765891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.766533 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:18.768518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:18.892191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1403/2797] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:18 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:09:18.893467 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.905912 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770d626a-e3a5-4f7c-86e0-5a241f1adc51 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.910662 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.911278 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.911564 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.911862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.912543 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:18.914181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:18.926176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1395/2798] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:18.929994 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.940232 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfbe4f-97eb-456d-b63b-bba7e058a8b8 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.946266 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231b2925-6cec-417d-99cd-d4c1526f2ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.946881 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b2925-6cec-417d-99cd-d4c1526f2ff7 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.947160 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b2925-6cec-417d-99cd-d4c1526f2ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.947431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231b2925-6cec-417d-99cd-d4c1526f2ff7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.948274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b2925-6cec-417d-99cd-d4c1526f2ff7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:18.949865 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231b2925-6cec-417d-99cd-d4c1526f2ff7 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:18.965101 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1404/2799] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:18.966239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:18.976805 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652d054f-4c6e-47c2-87d2-5f90533a38d9 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:18.981584 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:18.982241 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:18.982518 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:18.982790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] PATH_INFO: `/v3/projects/5bb352939e684e269dc5e4f69690eb50/users/49b6c291878348eea8a53a3f8aced7c7/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:18.997974 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bb352939e684e269dc5e4f69690eb50, user_id=49b6c291878348eea8a53a3f8aced7c7, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:19.000580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:19.203397 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1396/2800] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:18 2022] PUT /identity/v3/projects/5bb352939e684e269dc5e4f69690eb50/users/49b6c291878348eea8a53a3f8aced7c7/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:19.215936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83921-bfd2-44fc-be92-6b5fbe48bcd1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:19.226480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83921-bfd2-44fc-be92-6b5fbe48bcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:19.226480 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e83921-bfd2-44fc-be92-6b5fbe48bcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:19.247141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f1e83921-bfd2-44fc-be92-6b5fbe48bcd1 None None] MFA Rules not processed for user `49b6c291878348eea8a53a3f8aced7c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:19.248360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f1e83921-bfd2-44fc-be92-6b5fbe48bcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:19.317149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1405/2801] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:19 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:19.328143 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:19.330260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:19.368376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:19.389950 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9a59df-809d-4777-b08a-0d7733b30bb7 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:19.393607 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:19.394741 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:19.395020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:19.395294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:19.403929 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:19.405464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:19.415231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1397/2802] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:19 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:22.663470 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:22.663976 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:22.664086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:22.685751 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] MFA Rules not processed for user `b7d2ab8bd072488091fb5a94f59946d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:22.686694 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:22.743403 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1406/2803] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:22 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:23.093309 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:23.094754 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:23.154469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6259a8-aca5-44b9-88a1-4f6f0dfe7f11 service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:23.158076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:23.158674 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:23.158946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:23.159210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:23.161186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:23.170850 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:23.172254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7341fc8a-86cd-42bb-8fa7-368684daefcc service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:23.181189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1398/2804] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:23 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:09:32.735804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=76,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:33.177393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=37 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:38.523194 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:38.598876 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c57457-4bd5-4ab8-a762-5a9cd624adc3 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:38.605405 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:38.606059 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:38.606334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:38.606604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] PATH_INFO: `/v3/users/49b6c291878348eea8a53a3f8aced7c7` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:38.607936 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49b6c291878348eea8a53a3f8aced7c7)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:38.610029 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:38.835459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49b6c291878348eea8a53a3f8aced7c7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:38.836069 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49b6c291878348eea8a53a3f8aced7c7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:38.836687 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49b6c291878348eea8a53a3f8aced7c7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:38.845290 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49b6c291878348eea8a53a3f8aced7c7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:38.858699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49b6c291878348eea8a53a3f8aced7c7'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:38.930962 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1407/2805] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:38 2022] DELETE /identity/v3/users/49b6c291878348eea8a53a3f8aced7c7 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:38.935923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:38.936532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:38.936532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:38.956416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:38.958088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:39.022135 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1399/2806] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:39.032543 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:39.034028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:39.099789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c3263e-196a-4b8a-94ca-001b25ff4c49 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:39.103528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaf23f5-2849-4b4d-96ec-69fedfa4a857 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:39.104137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf23f5-2849-4b4d-96ec-69fedfa4a857 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:39.104444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf23f5-2849-4b4d-96ec-69fedfa4a857 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:39.104719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaf23f5-2849-4b4d-96ec-69fedfa4a857 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:39.114633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf23f5-2849-4b4d-96ec-69fedfa4a857 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:39.116084 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efaf23f5-2849-4b4d-96ec-69fedfa4a857 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:39.125910 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1408/2807] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:39.576987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:39.578597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:39.595556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9fe286-94b4-4710-90b3-9b9bed3da6ac None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:39.597507 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:39.598434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:39.598724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:39.599000 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] PATH_INFO: `/v3/projects/5bb352939e684e269dc5e4f69690eb50` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:39.601229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bb352939e684e269dc5e4f69690eb50)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:39.602290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:39.740014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bb352939e684e269dc5e4f69690eb50', 'request_id': 'req-e0909566-1b9c-48bc-94bc-b9339007a415'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:39.746845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bb352939e684e269dc5e4f69690eb50', 'request_id': 'req-e0909566-1b9c-48bc-94bc-b9339007a415'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:39.754536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bb352939e684e269dc5e4f69690eb50', 'request_id': 'req-e0909566-1b9c-48bc-94bc-b9339007a415'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:39.871555 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] The token cache is being invalidate because project 5bb352939e684e269dc5e4f69690eb50 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:39.871844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:39.872483 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e0909566-1b9c-48bc-94bc-b9339007a415 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:39.874057 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1400/2808] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:39 2022] DELETE /identity/v3/projects/5bb352939e684e269dc5e4f69690eb50 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:39.879508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:39.880073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:39.880073 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:39.899254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:39.900446 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:39.918426 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:39.963712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1409/2809] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:09:39.971955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:39.972951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:39.973215 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:39.993830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:40.000604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:40.003023 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1401/2810] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:40.007350 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.017230 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a69a7c-9e18-4df6-ad22-7a3c333fd5a4 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.022132 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.022737 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.023018 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.023292 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.024066 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:09:40.024378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.026068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.039227 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1410/2811] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:09:40.043592 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.053951 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81f6e13-2fa1-4e2b-9aaa-b7ca2662cf19 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.058736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.059364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.059645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.060046 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.060688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.062623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.159891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1402/2812] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:40 2022] POST /identity/v3/projects => generated 385 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:40.164788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.175048 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7161e6a-0e8d-48db-8620-d268cbe9bd04 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.234467 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.235094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.235550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.235845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.236474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.238571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.358854 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1411/2813] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:40 2022] POST /identity/v3/users => generated 293 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:40.363316 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.373943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88f4529-469c-44d3-8e4b-e55cfd340917 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.379817 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.380429 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.380709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.380984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.381671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.383245 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.393901 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1403/2814] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:40.399141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.411880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a810f203-92d5-4f35-80b1-9c9b2ebbed79 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.417635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509f2f24-aabb-4022-a6e2-2a2346d7e278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.418294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f2f24-aabb-4022-a6e2-2a2346d7e278 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.418581 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f2f24-aabb-4022-a6e2-2a2346d7e278 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.418855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509f2f24-aabb-4022-a6e2-2a2346d7e278 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.419540 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f2f24-aabb-4022-a6e2-2a2346d7e278 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.421260 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509f2f24-aabb-4022-a6e2-2a2346d7e278 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.435050 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1412/2815] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:09:40.440134 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.452480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e156686-c0ee-4180-8d8e-f0c3f22824eb admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.459519 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.460188 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.460469 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.460763 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] PATH_INFO: `/v3/projects/3184f1249a574f2c8997e156b535aa91/users/58f474208a1d4d16a0aa7a31555c945a/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.477003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3184f1249a574f2c8997e156b535aa91, user_id=58f474208a1d4d16a0aa7a31555c945a, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.479649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.541895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1404/2816] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:40 2022] PUT /identity/v3/projects/3184f1249a574f2c8997e156b535aa91/users/58f474208a1d4d16a0aa7a31555c945a/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:40.556759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.556759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.556759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.586438 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] MFA Rules not processed for user `58f474208a1d4d16a0aa7a31555c945a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:40.587836 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:40.670559 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1413/2817] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:40 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:40.684256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:40.686055 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:40.709918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:40.745620 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e92433b-0802-4aa0-b160-956dd7b6dd0e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:40.749250 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b576cbe-a82d-4794-8019-78deec37a378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:40.749857 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576cbe-a82d-4794-8019-78deec37a378 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:40.750130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576cbe-a82d-4794-8019-78deec37a378 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:40.750399 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b576cbe-a82d-4794-8019-78deec37a378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:40.762917 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576cbe-a82d-4794-8019-78deec37a378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:40.764385 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b576cbe-a82d-4794-8019-78deec37a378 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:40.774570 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1405/2818] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:40 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:49.415444 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:49.437949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:49.529342 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52869ac1-145b-44d8-bd83-b5eb8e0c8b07 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:49.537766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:49.538862 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:49.539365 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:49.539861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] PATH_INFO: `/v3/users/58f474208a1d4d16a0aa7a31555c945a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:49.542012 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58f474208a1d4d16a0aa7a31555c945a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:49.545815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:49.868620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58f474208a1d4d16a0aa7a31555c945a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:49.869514 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58f474208a1d4d16a0aa7a31555c945a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:49.870667 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58f474208a1d4d16a0aa7a31555c945a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:49.882226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58f474208a1d4d16a0aa7a31555c945a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:49.898731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58f474208a1d4d16a0aa7a31555c945a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:49.989698 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1414/2819] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:09:49 2022] DELETE /identity/v3/users/58f474208a1d4d16a0aa7a31555c945a => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:49.994491 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:49.994994 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:49.995085 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:50.018279 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:50.019776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:50.084642 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1406/2820] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:50.094672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:50.096097 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:50.170864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ccd0943-4b65-48f3-ab56-db5f8e06fc48 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:50.174545 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3609e330-d5ba-4563-8a2f-2e0f2f577188 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:50.175154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609e330-d5ba-4563-8a2f-2e0f2f577188 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:50.175435 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609e330-d5ba-4563-8a2f-2e0f2f577188 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:50.175711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3609e330-d5ba-4563-8a2f-2e0f2f577188 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:50.188237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609e330-d5ba-4563-8a2f-2e0f2f577188 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:50.189816 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3609e330-d5ba-4563-8a2f-2e0f2f577188 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:50.204991 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1415/2821] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:09:51.184078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:51.185728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:51.224545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c1983f-9000-4c79-9a2a-d1b1689ea6ee None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:51.235229 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:51.235376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:51.235580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:51.235941 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] PATH_INFO: `/v3/projects/3184f1249a574f2c8997e156b535aa91` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:51.237383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3184f1249a574f2c8997e156b535aa91)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:51.240053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:51.377810 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3184f1249a574f2c8997e156b535aa91', 'request_id': 'req-88e766b0-9bda-40d7-b437-4da7f6dd02d9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:51.386535 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3184f1249a574f2c8997e156b535aa91', 'request_id': 'req-88e766b0-9bda-40d7-b437-4da7f6dd02d9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:51.393671 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3184f1249a574f2c8997e156b535aa91', 'request_id': 'req-88e766b0-9bda-40d7-b437-4da7f6dd02d9'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:51.473342 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] The token cache is being invalidate because project 3184f1249a574f2c8997e156b535aa91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:09:51.473847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:51.475115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-88e766b0-9bda-40d7-b437-4da7f6dd02d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:09:51.480235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1407/2822] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:09:51 2022] DELETE /identity/v3/projects/3184f1249a574f2c8997e156b535aa91 => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:51.482983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:51.484151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:51.484433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:51.537863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:51.539157 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:51.578734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:51.611234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1416/2823] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:51.620509 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:51.620790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:51.621093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:51.674111 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:51.677634 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:51.689755 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1408/2824] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:51.691318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:51.702968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4030e53-cfbb-40da-8ca9-5290c05e0f5c None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:51.708384 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:51.709319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:51.709601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:51.709875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:51.710553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:09:51.710885 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:51.712858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:51.730334 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1417/2825] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:09:51.731571 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:51.742412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848b2677-093e-40d0-a80b-8aa82f45d5e6 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:51.752878 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:51.753647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:51.753932 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:51.754206 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:51.754886 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:51.757258 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:51.859113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1409/2826] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:09:51 2022] POST /identity/v3/projects => generated 399 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:51.863464 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:51.874843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c111e3-8f5d-40af-8d11-991322a275b1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:51.942897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:51.943617 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:51.943902 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:51.944177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:51.944828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:51.951191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:52.082424 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1418/2827] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:09:51 2022] POST /identity/v3/users => generated 300 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:09:52.094276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:52.096537 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f8bc00-5d0d-471e-b1c7-fb9b22ea33ae admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:52.103072 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:52.103707 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:52.103988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:52.104259 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:52.105230 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:52.106709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:52.119337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1410/2828] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:09:52.123745 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:52.136472 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4886f96a-2ff6-4213-a3c6-224f676fd759 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:52.142209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3133c-eb6d-4e40-b6e4-e7366f19fd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:52.142913 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3133c-eb6d-4e40-b6e4-e7366f19fd3d admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:52.143259 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3133c-eb6d-4e40-b6e4-e7366f19fd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:52.143620 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3133c-eb6d-4e40-b6e4-e7366f19fd3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:52.144336 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3133c-eb6d-4e40-b6e4-e7366f19fd3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:52.146706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c3133c-eb6d-4e40-b6e4-e7366f19fd3d admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:52.164579 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1419/2829] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:09:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:09:52.166284 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:52.181883 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b5aaf7a-9743-460b-ba6e-894ad19dd2d1 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:52.186732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:52.187494 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:52.188752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:52.189171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] PATH_INFO: `/v3/projects/16a70423a39b4c869faca8c65b703073/users/1d86dc1f78d14b389990a26c689a731a/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:52.192020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:09:52.207336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=16a70423a39b4c869faca8c65b703073, user_id=1d86dc1f78d14b389990a26c689a731a, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:52.212938 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:52.347762 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1411/2830] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:09:52 2022] PUT /identity/v3/projects/16a70423a39b4c869faca8c65b703073/users/1d86dc1f78d14b389990a26c689a731a/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:09:52.358445 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:52.358730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:52.359007 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:52.389195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] MFA Rules not processed for user `1d86dc1f78d14b389990a26c689a731a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:09:52.396228 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:52.465521 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1420/2831] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:09:52 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:09:52.477699 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:09:52.479423 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:09:52.546908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e64b104-c77a-42d3-8b0b-211744f309c3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:09:52.550552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38faf201-7710-4bb9-83de-954e251d22e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:09:52.552336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf201-7710-4bb9-83de-954e251d22e2 service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:09:52.552790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf201-7710-4bb9-83de-954e251d22e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:09:52.553282 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38faf201-7710-4bb9-83de-954e251d22e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:09:52.562756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf201-7710-4bb9-83de-954e251d22e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:09:52.564190 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38faf201-7710-4bb9-83de-954e251d22e2 service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:09:52.574244 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1412/2832] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:09:52 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:02.462086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:02.569893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:08.031013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:08.095008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f927bea-6290-4437-af61-7b5efdcc5007 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:08.099515 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:08.100209 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:08.100517 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:08.101318 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] PATH_INFO: `/v3/users/1d86dc1f78d14b389990a26c689a731a` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:08.102491 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d86dc1f78d14b389990a26c689a731a)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:08.104473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:08.371197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d86dc1f78d14b389990a26c689a731a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:08.371858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d86dc1f78d14b389990a26c689a731a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:08.372461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d86dc1f78d14b389990a26c689a731a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:08.379720 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d86dc1f78d14b389990a26c689a731a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:08.393278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d86dc1f78d14b389990a26c689a731a'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:08.473863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1421/2833] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:10:08 2022] DELETE /identity/v3/users/1d86dc1f78d14b389990a26c689a731a => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:08.479294 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:08.479756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:08.480050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:08.508841 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:08.510110 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:08.574899 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1413/2834] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:08.591307 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:08.592749 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:08.665450 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0744f239-ce04-4676-9dae-b5940375af79 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:08.669176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:08.669785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:08.670068 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:08.670341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:08.679688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:08.681175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:08.691439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1422/2835] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:09.265212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:09.265212 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:09.275730 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3360bc-3574-4b5f-a260-c438d7cfe3a7 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:09.281163 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:09.281998 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:09.282281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:09.282556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] PATH_INFO: `/v3/projects/16a70423a39b4c869faca8c65b703073` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:09.283843 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=16a70423a39b4c869faca8c65b703073)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:09.285792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:09.464821 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '16a70423a39b4c869faca8c65b703073', 'request_id': 'req-440636df-99f4-4bb9-85ca-4c98c39aa8ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.470499 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '16a70423a39b4c869faca8c65b703073', 'request_id': 'req-440636df-99f4-4bb9-85ca-4c98c39aa8ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.476990 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '16a70423a39b4c869faca8c65b703073', 'request_id': 'req-440636df-99f4-4bb9-85ca-4c98c39aa8ea'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.655706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:09.658081 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:09.671250 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527aeffa-0e0a-4dbf-b935-e8bef91d44e1 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:09.681390 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:09.684417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] The token cache is being invalidate because project 16a70423a39b4c869faca8c65b703073 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:10:09.684969 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.701582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-440636df-99f4-4bb9-85ca-4c98c39aa8ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.701582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1414/2836] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:10:09 2022] DELETE /identity/v3/projects/16a70423a39b4c869faca8c65b703073 => generated 0 bytes in 427 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:09.705389 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65107-3531-4bb6-9260-b52c614b92e7 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:09.705713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65107-3531-4bb6-9260-b52c614b92e7 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:09.706012 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65107-3531-4bb6-9260-b52c614b92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:09.727677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c9a65107-3531-4bb6-9260-b52c614b92e7 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:09.728957 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c9a65107-3531-4bb6-9260-b52c614b92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:09.732429 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:09.733108 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:09.733394 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:09.733674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] PATH_INFO: `/v3/users/4db176d0edeb4d2e8c438a0e675dbac1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:09.734813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4db176d0edeb4d2e8c438a0e675dbac1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:09.736857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:09.737563 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1415/2837] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:10:09.747828 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:09.764906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:09.764906 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:09.769587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:09.770866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:09.780061 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1416/2838] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:10:09.784340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:09.794567 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0550d2b4-0df6-4afd-8fa7-1ccd90a5181d None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:09.799724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:09.800326 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:09.804870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:09.804870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:09.804870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:10:09.804870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:09.804870 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:09.815866 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1417/2839] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 23 23:10:09.820195 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:09.830453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf32cf2-8c0c-42ae-a37e-cf68c9c0bbe5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:09.836445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:09.841630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:09.841630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:09.841630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:09.841630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:09.841630 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:09.950139 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1418/2840] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:10:09 2022] POST /identity/v3/projects => generated 377 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:10:09.954371 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:09.979821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4db176d0edeb4d2e8c438a0e675dbac1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.980505 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4db176d0edeb4d2e8c438a0e675dbac1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.981169 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4db176d0edeb4d2e8c438a0e675dbac1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:09.986678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:09.988670 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4db176d0edeb4d2e8c438a0e675dbac1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:10.001824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-ab4a7778-851b-4a31-a327-2c9eb46a0455 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4db176d0edeb4d2e8c438a0e675dbac1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:10.019595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe75fba8-7e06-4948-9bd7-3997b37fc736 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:10.024851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:10.025502 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.025783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.026088 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.026718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:10.028887 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:10.079850 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1423/2841] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:10:09 2022] DELETE /identity/v3/users/4db176d0edeb4d2e8c438a0e675dbac1 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:10.084624 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.084941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.085412 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.107697 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:10.117302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:10.117558 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1419/2842] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:10:09 2022] POST /identity/v3/users => generated 289 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:10:10.121199 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:10.122971 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:10.137874 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33cb5d-451b-4673-9c54-aae9aad21628 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:10.175927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1424/2843] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:10:10.187456 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:10.189028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:10.206426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:10.207280 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.207632 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.207985 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.208624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:10.210934 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:10.222147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1420/2844] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:10:10.226210 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:10.233276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:10.236603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835ee729-c964-44c9-af4f-a4996892f28a admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:10.241708 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:10.242597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.247875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.247875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.247875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:10.247875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:10.256461 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1421/2845] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:10:10.257882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7bec86-28f6-4c91-839f-46552394989f None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:10.261252 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:10.262064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3a3953-9221-4c3b-9c36-587a2ca97409 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:10.262671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3953-9221-4c3b-9c36-587a2ca97409 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.262953 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3953-9221-4c3b-9c36-587a2ca97409 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.263221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3a3953-9221-4c3b-9c36-587a2ca97409 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.271953 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeeceea9-d797-4d62-80bd-e72083fcaa0d admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:10.274845 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a3953-9221-4c3b-9c36-587a2ca97409 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:10.276381 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3a3953-9221-4c3b-9c36-587a2ca97409 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:10.277844 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:10.278510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.282334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.282334 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] PATH_INFO: `/v3/projects/9e5fdd0ced9b4bb3af81a136afaca202/users/51af1e82755e449993e110db493b689b/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.287649 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1425/2846] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:10:10.295908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e5fdd0ced9b4bb3af81a136afaca202, user_id=51af1e82755e449993e110db493b689b, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:10.298688 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:10.446453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1422/2847] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:10:10 2022] PUT /identity/v3/projects/9e5fdd0ced9b4bb3af81a136afaca202/users/51af1e82755e449993e110db493b689b/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:10:10.452100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.452388 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.452669 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.473159 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:10.475286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:10.537129 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1426/2848] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:10.548752 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:10.551053 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:10.624099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abea392-622b-4df7-8e84-60b13dc4dd92 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:10.628330 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7289be51-a73a-419e-a41d-1946e796a125 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:10.628962 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289be51-a73a-419e-a41d-1946e796a125 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:10.629336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289be51-a73a-419e-a41d-1946e796a125 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:10.629619 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7289be51-a73a-419e-a41d-1946e796a125 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:10.641673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289be51-a73a-419e-a41d-1946e796a125 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:10.643124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7289be51-a73a-419e-a41d-1946e796a125 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:10.654859 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1423/2849] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:11.043206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:11.045287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:11.059987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3582d2a8-f6f1-4b32-b6c3-f2e2155a6e5b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:11.066706 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:11.067813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.068104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.068416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] PATH_INFO: `/v3/projects/343c907da4ea423e8f1ef6c13bd576e8` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.069726 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] RBAC: Authorizing `identity:delete_project(project_id=343c907da4ea423e8f1ef6c13bd576e8)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:11.071671 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:11.223473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '343c907da4ea423e8f1ef6c13bd576e8', 'request_id': 'req-be03213a-779c-43ad-ad02-d84bbcad5a5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:11.230711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '343c907da4ea423e8f1ef6c13bd576e8', 'request_id': 'req-be03213a-779c-43ad-ad02-d84bbcad5a5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:11.313712 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '343c907da4ea423e8f1ef6c13bd576e8', 'request_id': 'req-be03213a-779c-43ad-ad02-d84bbcad5a5f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:11.327177 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] The token cache is being invalidate because project 343c907da4ea423e8f1ef6c13bd576e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81612) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:10:11.327600 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:11.328263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-be03213a-779c-43ad-ad02-d84bbcad5a5f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:10:11.329897 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1427/2850] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:10:11 2022] DELETE /identity/v3/projects/343c907da4ea423e8f1ef6c13bd576e8 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:11.334854 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.335946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.335946 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.357551 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:11.358781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:11.398511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:11.428126 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1424/2851] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:11.436229 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.436554 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.436829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.456974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:11.458262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:11.467153 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1428/2852] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:11.471776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:11.482776 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f390614-70af-4ca7-8840-1bb5e1bc8088 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:11.487647 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:11.488267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.491529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.491529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] PATH_INFO: `/v3/domains` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.491529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81613) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:10:11.491529 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:11.491847 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:11.503787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1425/2853] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:10:11.508468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:11.518402 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb7d09-6a92-48b9-9a4f-90c1d2456424 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:11.524046 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:11.524823 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.525172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.531754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.531754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:11.531754 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:11.647348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1429/2854] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:10:11 2022] POST /identity/v3/projects => generated 371 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:10:11.652354 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:11.663627 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f85da7-c363-4d18-bf0a-c7edbc8c4838 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:11.723825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:11.724351 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.724633 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.724905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.725603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:11.727668 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:11.892117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1426/2855] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:10:11 2022] POST /identity/v3/users => generated 286 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:10:11.918255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:11.918255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f717b0-341f-4f79-b869-94c126bcd4da admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:11.918255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:11.918255 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.919676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.919676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.919676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:11.919676 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:11.929873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1430/2856] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:10:11.933788 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:11.944104 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615ad74-9517-438a-977e-966058147e0e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:11.950235 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bccab0-2908-4142-95f9-4f6d81e8bd89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:11.950840 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bccab0-2908-4142-95f9-4f6d81e8bd89 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:11.951147 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bccab0-2908-4142-95f9-4f6d81e8bd89 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:11.951422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bccab0-2908-4142-95f9-4f6d81e8bd89 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:11.952070 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bccab0-2908-4142-95f9-4f6d81e8bd89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:11.953848 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bccab0-2908-4142-95f9-4f6d81e8bd89 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:11.966639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1427/2857] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:10:11.971022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:12.021192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee15edb-82e6-484b-8f87-c87ed9bbb6ae admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:12.021192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:12.021192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:12.021192 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:12.021578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] PATH_INFO: `/v3/projects/aa51baae7e1f4d318a3c9607d5cb715d/users/daf49a1b3de0478a939694287535607f/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:12.021578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa51baae7e1f4d318a3c9607d5cb715d, user_id=daf49a1b3de0478a939694287535607f, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:12.021578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:12.021578 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:12.172044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1431/2858] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:10:11 2022] PUT /identity/v3/projects/aa51baae7e1f4d318a3c9607d5cb715d/users/daf49a1b3de0478a939694287535607f/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:12.177099 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d3e29-5225-4d34-b790-e5e2c9d55e02 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:12.177393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d3e29-5225-4d34-b790-e5e2c9d55e02 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:12.177670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6d3e29-5225-4d34-b790-e5e2c9d55e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:12.198414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-4d6d3e29-5225-4d34-b790-e5e2c9d55e02 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:12.199673 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-4d6d3e29-5225-4d34-b790-e5e2c9d55e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:12.261517 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1428/2859] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:12.271320 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:12.273171 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:12.340668 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982053ee-314d-44d1-b6ab-089287e8c392 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:12.344742 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:12.345383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:12.345656 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:12.345927 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:12.357372 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:12.358889 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:12.371096 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1432/2860] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:17.835923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:17.837204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:17.837649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:17.871853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] MFA Rules not processed for user `51af1e82755e449993e110db493b689b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:17.873361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:17.889505 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:17.956113 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1429/2861] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:17.961844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:17.964161 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:17.978682 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b9673-6d81-4f63-a99c-29aa3c45cdb6 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:17.984302 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:17.985237 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:17.985516 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:17.985957 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] PATH_INFO: `/v3/projects` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:17.986651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:17.988672 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:18.125270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1433/2862] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:10:17 2022] POST /identity/v3/projects => generated 377 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:10:18.129087 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:18.142454 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a595f293-8ae6-49ac-a1ab-9b0bc9e7ca7e None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:18.201895 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:18.202569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:18.202826 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:18.203107 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] PATH_INFO: `/v3/users` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:18.203777 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:18.205773 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:18.315124 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1430/2863] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:10:18 2022] POST /identity/v3/users => generated 295 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 23 23:10:18.319121 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:18.332701 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8129fc2-63cc-413f-8c56-6838406c1967 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:18.339035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:18.339665 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:18.339945 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:18.340216 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:18.340868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:18.342539 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:18.357062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1434/2864] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:10:18.360812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:18.373728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c29583fa-fce3-4a86-b273-931cfe56bb62 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:18.383601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:18.383601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:18.383601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:18.383601 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] PATH_INFO: `/v3/projects/dd212f561fa147ce9d8e64182550fb2e/users/b5ae8f3a7f4944dfbcd33904bee6a9ce/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:18.396824 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd212f561fa147ce9d8e64182550fb2e, user_id=b5ae8f3a7f4944dfbcd33904bee6a9ce, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:18.399722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:18.517983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1431/2865] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:10:18 2022] PUT /identity/v3/projects/dd212f561fa147ce9d8e64182550fb2e/users/b5ae8f3a7f4944dfbcd33904bee6a9ce/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:10:18.521774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:18.536734 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555aa1ed-3322-4e2f-9e74-0921bf06f378 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:18.650086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2300fa1-76ea-48ad-a0ff-772b2c9de646 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:18.650802 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2300fa1-76ea-48ad-a0ff-772b2c9de646 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:18.651604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2300fa1-76ea-48ad-a0ff-772b2c9de646 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:18.651604 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2300fa1-76ea-48ad-a0ff-772b2c9de646 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:18.652258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2300fa1-76ea-48ad-a0ff-772b2c9de646 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:18.654232 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2300fa1-76ea-48ad-a0ff-772b2c9de646 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:18.670185 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1435/2866] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:18 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:10:18.674337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:18.688751 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d1972-17e1-4a1d-bd1c-263def149ad0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:18.694725 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:18.695785 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:18.696138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:18.696488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] PATH_INFO: `/v3/projects/dd212f561fa147ce9d8e64182550fb2e/users/b5ae8f3a7f4944dfbcd33904bee6a9ce/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:18.700495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd212f561fa147ce9d8e64182550fb2e, user_id=b5ae8f3a7f4944dfbcd33904bee6a9ce, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:18.703735 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:18.728376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1432/2867] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:10:18 2022] PUT /identity/v3/projects/dd212f561fa147ce9d8e64182550fb2e/users/b5ae8f3a7f4944dfbcd33904bee6a9ce/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:19.786571 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:19.787526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:19.788024 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:19.828783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] MFA Rules not processed for user `daf49a1b3de0478a939694287535607f`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:19.831222 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:19.906328 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:19.958128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1436/2868] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:19.958320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:20.078364 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b23b003-0891-44e5-a88f-f021909523e3 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:20.085030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:20.086193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:20.086484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:20.086853 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:20.087524 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:20.089740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:20.106298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:20.230945 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1433/2869] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:10:19 2022] POST /identity/v3/projects => generated 373 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:10:20.235422 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:20.248863 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188f1a7e-c944-498c-af93-4c905ee912e1 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:20.317766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:20.318406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:20.318736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:20.319258 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:20.319718 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:20.322013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:20.428301 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1437/2870] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:10:20 2022] POST /identity/v3/users => generated 293 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:10:20.433222 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:20.446909 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f835aa9-08c2-4815-b6b3-b47a5644858e admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:20.451639 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:20.457391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:20.457391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:20.457391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:20.457391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:20.457391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:20.467943 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1434/2871] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:10:20.471949 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:20.485642 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b42f3bf-6f5e-4bc5-88f4-2fab4c0d02cf admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:20.493995 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:20.497262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:20.497262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:20.497262 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] PATH_INFO: `/v3/projects/30a63975c48646f4ad8f98da2256028c/users/170b0fca9092474fa004b5209d2d96f1/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:20.516812 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a63975c48646f4ad8f98da2256028c, user_id=170b0fca9092474fa004b5209d2d96f1, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:20.520016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:20.730321 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1438/2872] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:10:20 2022] PUT /identity/v3/projects/30a63975c48646f4ad8f98da2256028c/users/170b0fca9092474fa004b5209d2d96f1/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:20.730457 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:20.744713 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130596dd-aaf8-48d5-b90d-2eef93c8a1c0 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:20.896539 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0eaf86-78f8-4dcf-b866-9b64b4be0c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:20.898473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eaf86-78f8-4dcf-b866-9b64b4be0c2b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:20.898600 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eaf86-78f8-4dcf-b866-9b64b4be0c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:20.898993 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0eaf86-78f8-4dcf-b866-9b64b4be0c2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:20.900323 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eaf86-78f8-4dcf-b866-9b64b4be0c2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:20.904749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0eaf86-78f8-4dcf-b866-9b64b4be0c2b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:20.938864 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:20.939422 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1435/2873] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 23 23:10:20.958203 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0730bef-fadf-4075-a8b0-13f2d37e5bd6 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:20.967348 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:20.974049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:20.974049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:20.974049 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] PATH_INFO: `/v3/projects/30a63975c48646f4ad8f98da2256028c/users/170b0fca9092474fa004b5209d2d96f1/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:20.986214 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30a63975c48646f4ad8f98da2256028c, user_id=170b0fca9092474fa004b5209d2d96f1, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:20.990395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:21.029276 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1439/2874] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:10:20 2022] PUT /identity/v3/projects/30a63975c48646f4ad8f98da2256028c/users/170b0fca9092474fa004b5209d2d96f1/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 91 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:10:26.365851 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daa000-c0b5-4420-ad61-3902a9404f8f None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:26.366336 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daa000-c0b5-4420-ad61-3902a9404f8f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:26.366411 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1daa000-c0b5-4420-ad61-3902a9404f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:26.415809 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e1daa000-c0b5-4420-ad61-3902a9404f8f None None] MFA Rules not processed for user `b5ae8f3a7f4944dfbcd33904bee6a9ce`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:26.417583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e1daa000-c0b5-4420-ad61-3902a9404f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:26.529393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1436/2875] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:26 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:26.547774 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:26.548908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:26.707638 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f5184-9c95-4603-9cff-306249a95d48 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:26.726102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:26.726815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:26.727099 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:26.727362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:26.772853 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:26.803657 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:26.808519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:26.828907 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1440/2876] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:26 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:28.443882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5255c7-aac8-4642-aaec-1c56cd2afbff None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:28.443882 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5255c7-aac8-4642-aaec-1c56cd2afbff None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:28.444536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5255c7-aac8-4642-aaec-1c56cd2afbff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:28.477313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-db5255c7-aac8-4642-aaec-1c56cd2afbff None None] MFA Rules not processed for user `170b0fca9092474fa004b5209d2d96f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:28.478540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-db5255c7-aac8-4642-aaec-1c56cd2afbff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:28.558689 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:28.640875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1437/2877] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:28 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:28.653739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:28.655603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:28.698595 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fda617-06c0-4041-a01e-c5a151a56386 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:28.699875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:28.700488 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:28.700765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:28.701128 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:28.773977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:28.776261 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:28.792465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1441/2878] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:28 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:32.158018 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726b747-2c2d-42d0-b172-171b81d873ae None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:32.158470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726b747-2c2d-42d0-b172-171b81d873ae None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:32.158903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d726b747-2c2d-42d0-b172-171b81d873ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:32.182498 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-d726b747-2c2d-42d0-b172-171b81d873ae None None] MFA Rules not processed for user `daf49a1b3de0478a939694287535607f`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:10:32.183635 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-d726b747-2c2d-42d0-b172-171b81d873ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:32.200480 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1438/2879] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:10:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:10:38.786935 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=50 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:41.792838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:10:41.794393 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:41.894461 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6cb0142-f6b3-400c-a32d-9e3b18d3c0c5 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:10:41.900923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:10:41.901635 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:10:41.901857 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:10:41.902224 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:10:41.903988 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:10:41.918186 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:10:41.919815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:10:41.931503 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1442/2880] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:10:41 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:10:42.175581 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=17 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:10:51.926206 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=41 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:11:06.880171 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:11:06.880742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:11:06.880742 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:11:06.899819 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] MFA Rules not processed for user `51af1e82755e449993e110db493b689b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:11:06.901041 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:11:06.908384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1439/2881] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:11:06 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:11:16.894624 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:11:29.798294 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:11:29.854570 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfa605b-b469-44f5-a05b-79bd20a64f27 service placement] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:11:29.859288 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d02144b-5648-4609-a760-bc86709d22af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:11:29.859900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02144b-5648-4609-a760-bc86709d22af service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:11:29.860178 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02144b-5648-4609-a760-bc86709d22af service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:11:29.860448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d02144b-5648-4609-a760-bc86709d22af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:11:29.869983 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02144b-5648-4609-a760-bc86709d22af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:11:29.872411 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d02144b-5648-4609-a760-bc86709d22af service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:11:29.882431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1443/2882] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:11:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:11:33.126130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:11:33.136308 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b89bc6-5c74-4932-8eb5-456a3e5405a2 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:11:33.140299 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:11:33.145669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:11:33.145669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:11:33.145669 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:11:33.151523 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:11:33.153464 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:11:33.163427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1440/2883] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:11:33 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:11:39.878036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=26 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:11:43.159091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:11:51.273354 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:11:51.273848 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:11:51.273941 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:11:51.294987 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] MFA Rules not processed for user `b7d2ab8bd072488091fb5a94f59946d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:11:51.296187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:11:51.350768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1444/2884] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:11:51 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:12:01.344111 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=25 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:12:32.545456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:12:32.546779 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:12:32.578811 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc597786-19ee-4031-aeb5-24aedee2458f service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:12:32.582459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:12:32.583054 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:12:32.583331 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:12:32.583595 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:12:32.585345 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:12:32.594908 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:12:32.596366 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:12:32.604276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1441/2885] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:12:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:12:42.600724 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=15 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:13:23.467477 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:13:23.484588 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6530a36-d5bd-4acc-a658-8b6034ebd9e2 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:13:23.490966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:13:23.491627 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:23.491950 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:23.492263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:23.494249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:23.506191 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:13:23.507705 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:13:23.516793 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1445/2886] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:13:23 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:13:26.399931 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:13:26.401488 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:26.467842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bbd6a2-8794-4a00-bc20-b5a9b2ccf129 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:13:26.471458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:13:26.472059 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:26.472338 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:26.472612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:26.474352 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:26.488122 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:13:26.489582 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:13:26.501193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1442/2887] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:13:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:13:33.512576 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=8 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:13:35.925058 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:13:35.926603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:35.939416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2d332b-75a3-4368-bb32-268e509581bc service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:13:35.942986 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:13:35.943591 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:35.943868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:35.944139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:35.945815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:35.960275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:13:35.960497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30b336-83d9-48d4-9ef9-9ab742bbd845 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:13:35.971873 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1446/2888] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:13:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:13:36.496196 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=30 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:13:42.598260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:13:42.609711 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23c88b4-3a4b-4a84-a56d-0209b32c385b service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:13:42.615496 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:13:42.616093 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:42.616367 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:42.616637 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:42.618484 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:42.630954 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:13:42.632444 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:13:42.643816 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1443/2889] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:13:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:13:45.966829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=6 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:13:48.683716 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:48.684324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:48.684324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:48.704242 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] MFA Rules not processed for user `b7d2ab8bd072488091fb5a94f59946d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:13:48.705493 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:48.712690 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1447/2890] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:13:48 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:13:49.468117 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:13:49.469645 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:49.485193 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bc09a3-5a89-4b65-a491-56189e44eead service nova] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:13:49.492530 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:13:49.493298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:49.493720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:49.494101 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:49.496174 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:49.508836 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:13:49.510497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272b4a65-7959-427a-badf-5d9624459883 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:13:49.521091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1444/2891] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:13:49 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:13:54.682861 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:13:54.698847 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaac11c-6e97-4cd1-b624-c2bcb3bd4426 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:13:54.705061 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:13:54.705893 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:54.706319 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:54.706753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:54.708688 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:54.721843 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:13:54.724013 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:13:54.747143 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1448/2892] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:13:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:13:58.690806 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3f0a-2770-4037-ab21-7bc249a8430a None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:13:58.691265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3f0a-2770-4037-ab21-7bc249a8430a None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:13:58.691361 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d5e3f0a-2770-4037-ab21-7bc249a8430a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:13:58.711287 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8d5e3f0a-2770-4037-ab21-7bc249a8430a None None] MFA Rules not processed for user `51af1e82755e449993e110db493b689b`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:13:58.712553 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8d5e3f0a-2770-4037-ab21-7bc249a8430a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:13:58.720348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1445/2893] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:13:58 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:14:00.575207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:14:00.586607 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20745b78-97e4-4b88-ae03-b6cb030294de service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:14:00.593139 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:14:00.593752 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:00.594030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:00.594311 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:00.596207 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:14:00.609643 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:14:00.611062 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bbc08-1fd9-4335-a4b2-8bbd24598c05 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:14:00.622715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1449/2894] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:14:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:14:08.706343 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=15 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:14:10.617855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=15 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:14:19.103495 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d83d9-6488-495f-ab24-053084d0b14f None None] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:19.104039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d83d9-6488-495f-ab24-053084d0b14f None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:19.104039 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d83d9-6488-495f-ab24-053084d0b14f None None] PATH_INFO: `/` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:19.109003 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1446/2895] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 23:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 23:14:19.112788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb27ef7-9fec-4984-b99f-a5ff4644126b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:19.113275 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb27ef7-9fec-4984-b99f-a5ff4644126b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:19.113383 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb27ef7-9fec-4984-b99f-a5ff4644126b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:19.185278 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-8bb27ef7-9fec-4984-b99f-a5ff4644126b None None] MFA Rules not processed for user `b05fdc39b370474fa59a63b4b0c7748a`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:14:19.186522 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-8bb27ef7-9fec-4984-b99f-a5ff4644126b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:14:19.193341 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1450/2896] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 23:14:19 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:14:19.207181 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d83d9-6488-495f-ab24-053084d0b14f None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:14:19.219014 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d83d9-6488-495f-ab24-053084d0b14f None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:14:19.222609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6d7b9-24c5-49c2-97c2-cfe43c3730ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:14:19.223205 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d7b9-24c5-49c2-97c2-cfe43c3730ce service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:19.223477 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d7b9-24c5-49c2-97c2-cfe43c3730ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:19.223766 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6d7b9-24c5-49c2-97c2-cfe43c3730ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:19.231125 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6d7b9-24c5-49c2-97c2-cfe43c3730ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:14:19.233591 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6d7b9-24c5-49c2-97c2-cfe43c3730ce service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:14:19.241254 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1447/2897] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:14:19 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:14:19.323795 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb746526-653d-4fbb-b0dc-0e6f7a0735db None None] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:19.324078 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb746526-653d-4fbb-b0dc-0e6f7a0735db None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:19.324353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb746526-653d-4fbb-b0dc-0e6f7a0735db None None] PATH_INFO: `/` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:19.325794 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1451/2898] 10.222.0.73 () {54 vars in 921 bytes} [Tue Aug 23 23:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 23 23:14:19.331405 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:19.331685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:19.331980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:19.371548 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] MFA Rules not processed for user `059f72c1793e492fb052bebbf29b5bec`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:14:19.372820 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:14:19.382532 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1448/2899] 10.222.0.73 () {58 vars in 1018 bytes} [Tue Aug 23 23:14:19 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:14:19.393498 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb746526-653d-4fbb-b0dc-0e6f7a0735db None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:14:19.405197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb746526-653d-4fbb-b0dc-0e6f7a0735db None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:14:19.409158 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:14:19.409760 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:14:19.410044 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:14:19.410316 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:14:19.420175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:14:19.422226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:14:19.433176 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1452/2900] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:14:19 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:14:29.369161 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=15 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:14:29.428123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=23 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:15:42.980450 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:15:42.992536 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c41b0d-7ea6-4bfd-ae8e-401451af0a24 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:15:42.998237 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:15:43.004794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:15:43.004794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:15:43.004794 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:15:43.008722 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:15:43.010468 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:15:43.020928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1449/2901] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:15:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:15:53.016340 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:15:56.807197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:15:56.817104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cba87-1f2a-41bb-8a51-2707d3b30154 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:15:56.820820 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:15:56.821479 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:15:56.821759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:15:56.822030 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:15:56.823804 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:15:56.888287 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:15:56.913223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:15:56.913223 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1453/2902] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:15:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:00.244611 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:00.262322 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbcff54f-b386-4325-a283-f120ce8e8a2e service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:00.268616 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:00.269285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:00.269588 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:00.269864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:00.271686 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:00.285839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:00.287677 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:00.298207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1450/2903] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:00.440021 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:00.441622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:00.454894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39ca12a-baea-405c-ba69-c89e03d77f69 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:00.459651 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:00.460327 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:00.465788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:00.465788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] PATH_INFO: `/v3/users/b7d2ab8bd072488091fb5a94f59946d3` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:00.465788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7d2ab8bd072488091fb5a94f59946d3)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:00.465788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:00.705088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7d2ab8bd072488091fb5a94f59946d3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:00.705759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7d2ab8bd072488091fb5a94f59946d3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:00.706373 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7d2ab8bd072488091fb5a94f59946d3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:00.713855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7d2ab8bd072488091fb5a94f59946d3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:00.727824 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-92d64cee-95cb-460a-a9d5-cfc4df29a4d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7d2ab8bd072488091fb5a94f59946d3'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:00.832509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1454/2904] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:16:00 2022] DELETE /identity/v3/users/b7d2ab8bd072488091fb5a94f59946d3 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:01.090791 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:01.102670 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9be168-0c47-441d-a93b-03fe5beb31d0 service placement] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:01.162138 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:01.162772 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:01.163075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:01.163349 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] PATH_INFO: `/v3/projects/226fb1c97e3f4e87a47326dae7fb262d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:01.164459 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=226fb1c97e3f4e87a47326dae7fb262d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:01.166416 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:01.304653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '226fb1c97e3f4e87a47326dae7fb262d', 'request_id': 'req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:01.310839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '226fb1c97e3f4e87a47326dae7fb262d', 'request_id': 'req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:01.318219 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '226fb1c97e3f4e87a47326dae7fb262d', 'request_id': 'req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:01.437580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] The token cache is being invalidate because project 226fb1c97e3f4e87a47326dae7fb262d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:16:01.437865 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:01.438482 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-1a8204c9-c4e3-43fe-888b-1cd7c063cf41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:01.439984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1451/2905] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:16:01 2022] DELETE /identity/v3/projects/226fb1c97e3f4e87a47326dae7fb262d => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:01.447553 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:01.448083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:01.448083 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:01.501942 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:01.503187 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:01.544875 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:01.563787 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1455/2906] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:16:01.571770 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:01.572050 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:01.572321 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:01.593613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:01.594830 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:01.602834 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1452/2907] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:01.607271 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:01.617550 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5459c83-6ff6-4355-94e8-ba9856446f4b None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:01.622362 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:01.624035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:01.624035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:01.624035 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:01.624401 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:16:01.624704 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:01.626397 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:01.643175 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1456/2908] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:16:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:16:01.647613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:01.657398 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e9f440-f5e9-49eb-9d67-6fdb21a8d714 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:01.662756 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:01.663424 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:01.663706 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:01.663973 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:01.664583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:01.666497 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:01.817885 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1453/2909] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:16:01 2022] POST /identity/v3/projects => generated 383 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:16:01.821974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:01.832844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca693d7c-5c2e-442a-939b-075485c12780 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:01.891880 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:01.892485 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:01.892765 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:01.893064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:01.893696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:01.895622 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:01.992713 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1457/2910] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:16:01 2022] POST /identity/v3/users => generated 292 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:16:01.996980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.008265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdbe32c-6b7a-4382-9aee-bb71cdcd5777 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.012910 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.014583 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.014812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.015084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.015774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.017412 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.028328 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1454/2911] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:16:02.033896 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.044666 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efe4a08-9466-4f6e-983b-9d962ed3759f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.049728 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea582fb-5451-4560-b027-237751d3be47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.050337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea582fb-5451-4560-b027-237751d3be47 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.050621 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea582fb-5451-4560-b027-237751d3be47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.050894 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea582fb-5451-4560-b027-237751d3be47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.051542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea582fb-5451-4560-b027-237751d3be47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.053179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea582fb-5451-4560-b027-237751d3be47 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.064188 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1458/2912] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:16:02.068514 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.079186 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03f233-e811-4af3-bcb3-f2edf83c214b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.083926 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.084561 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.084839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.085168 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] PATH_INFO: `/v3/projects/52e9867214904941a91e25b74ce2e712/users/81ebc34ef9774b20853684f3270f7dec/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.099410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] RBAC: Authorizing `identity:create_grant(project_id=52e9867214904941a91e25b74ce2e712, user_id=81ebc34ef9774b20853684f3270f7dec, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.102103 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.181432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1455/2913] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:16:02 2022] PUT /identity/v3/projects/52e9867214904941a91e25b74ce2e712/users/81ebc34ef9774b20853684f3270f7dec/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:02.194448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.194743 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.195028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.224364 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] MFA Rules not processed for user `81ebc34ef9774b20853684f3270f7dec`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:02.225650 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:02.295246 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1459/2914] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:02 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:02.300144 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.314064 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:02.362434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc15fd5-4255-492a-862f-e88a3d8bd31b admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.367108 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.367712 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.368091 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.368376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.369037 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.370982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.503613 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1456/2915] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:16:02 2022] POST /identity/v3/projects => generated 383 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:16:02.507661 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.518442 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b4ffd-f7e7-4dd6-812d-f85ac6b80f90 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.558715 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:02.576378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.576981 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.577286 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.577558 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.578181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.580100 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.728172 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1460/2916] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:16:02 2022] POST /identity/v3/users => generated 298 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:16:02.732372 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.744307 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0eef62-e691-42a0-bd5b-46cd7574e0a4 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.748940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.749623 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.750042 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.750325 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.751030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.752617 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.763290 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1457/2917] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:16:02.767759 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.777868 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713fc7c1-312f-4cfa-9e7b-068f8fa176f1 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.783181 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.783838 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.784117 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.784419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] PATH_INFO: `/v3/projects/b056acb21b0b4f818327ca72dbd31684/users/05ca5b336aa44b1180b569205a66e1a6/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.800699 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b056acb21b0b4f818327ca72dbd31684, user_id=05ca5b336aa44b1180b569205a66e1a6, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.803459 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.913583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1461/2918] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:16:02 2022] PUT /identity/v3/projects/b056acb21b0b4f818327ca72dbd31684/users/05ca5b336aa44b1180b569205a66e1a6/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:02.917876 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:02.927988 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030b5e18-b99f-48b9-aa94-153310d8fed5 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:02.984577 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0640525-f9be-41f6-b87e-76fc442e9221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:02.985051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0640525-f9be-41f6-b87e-76fc442e9221 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:02.985339 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0640525-f9be-41f6-b87e-76fc442e9221 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:02.985608 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0640525-f9be-41f6-b87e-76fc442e9221 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:02.986240 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0640525-f9be-41f6-b87e-76fc442e9221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:02.987825 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0640525-f9be-41f6-b87e-76fc442e9221 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:02.997651 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1458/2919] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:16:02 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:16:03.002299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:03.012497 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabdcc7a-93c3-4f2a-888f-fbe30d6fcb14 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:03.018201 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:03.018860 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:03.019141 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:03.019416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] PATH_INFO: `/v3/projects/b056acb21b0b4f818327ca72dbd31684/users/05ca5b336aa44b1180b569205a66e1a6/roles/1e8252b04575454e9d3dc7f3cb1dc64c` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:03.021374 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b056acb21b0b4f818327ca72dbd31684, user_id=05ca5b336aa44b1180b569205a66e1a6, role_id=1e8252b04575454e9d3dc7f3cb1dc64c)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:03.024002 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:03.036509 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1462/2920] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:16:03 2022] PUT /identity/v3/projects/b056acb21b0b4f818327ca72dbd31684/users/05ca5b336aa44b1180b569205a66e1a6/roles/1e8252b04575454e9d3dc7f3cb1dc64c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:03.047489 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:03.047783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:03.048092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:03.076114 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] MFA Rules not processed for user `05ca5b336aa44b1180b569205a66e1a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:03.077456 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:03.147417 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1459/2921] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:03 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:16:03.160008 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:03.161877 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:03.221487 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d0994-595e-44fa-96d5-b7677b1b4d0a admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:03.225235 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:03.225844 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:03.226154 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:03.226423 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:03.236026 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:03.237528 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:03.247064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1463/2922] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:03 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:03.431100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:03.443324 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb4f80b-18d3-4ee1-98cc-f0576a075ab5 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:03.447153 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5ee44-9fa0-41fd-9b06-c7f6e9a9536a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:03.447764 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5ee44-9fa0-41fd-9b06-c7f6e9a9536a service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:03.448044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5ee44-9fa0-41fd-9b06-c7f6e9a9536a service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:03.448317 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5ee44-9fa0-41fd-9b06-c7f6e9a9536a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:03.454400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:03.505955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5ee44-9fa0-41fd-9b06-c7f6e9a9536a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:03.507288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5ee44-9fa0-41fd-9b06-c7f6e9a9536a service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:03.517029 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1460/2923] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:03.985768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:04.052102 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6320ea1b-f37e-4e54-979c-791d8968eb25 service cinder] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:04.056856 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:04.057603 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:04.057919 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:04.058195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] PATH_INFO: `/v3/users/81ebc34ef9774b20853684f3270f7dec` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:04.059345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] RBAC: Authorizing `identity:delete_user(user_id=81ebc34ef9774b20853684f3270f7dec)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:04.061345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:04.267080 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=90,keystone:INSERT=5,keystone:DELETE=5 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:04.275131 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81ebc34ef9774b20853684f3270f7dec'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:04.275800 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81ebc34ef9774b20853684f3270f7dec'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:04.276406 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81ebc34ef9774b20853684f3270f7dec'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:04.284088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81ebc34ef9774b20853684f3270f7dec'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:04.295675 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81ebc34ef9774b20853684f3270f7dec'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:04.409465 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1464/2924] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:16:03 2022] DELETE /identity/v3/users/81ebc34ef9774b20853684f3270f7dec => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:16:04.414313 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:04.414597 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:04.414869 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:04.435238 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:04.436427 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:04.496001 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1461/2925] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:04.505974 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:04.507408 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:04.565840 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cf161f-be77-4c5c-a27c-b7e501c5c78f admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:04.573150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:04.573150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:04.573150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:04.573150 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:04.580828 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:04.582295 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:04.591908 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1465/2926] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:05.086121 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:05.087603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:05.100470 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5617f70-54d8-41cd-b839-679a709bd7fa None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:05.107276 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:05.107939 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:05.111097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:05.111097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] PATH_INFO: `/v3/projects/52e9867214904941a91e25b74ce2e712` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:05.111097 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=52e9867214904941a91e25b74ce2e712)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:05.113393 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:05.251787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '52e9867214904941a91e25b74ce2e712', 'request_id': 'req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.257320 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '52e9867214904941a91e25b74ce2e712', 'request_id': 'req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.263458 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '52e9867214904941a91e25b74ce2e712', 'request_id': 'req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.369404 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] The token cache is being invalidate because project 52e9867214904941a91e25b74ce2e712 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:16:05.369603 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.370231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.371736 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1462/2927] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:16:05 2022] DELETE /identity/v3/projects/52e9867214904941a91e25b74ce2e712 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:05.376378 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:05.386494 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918aec3-6051-439c-bcb6-a5613ceeaa52 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:05.443831 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:05.444458 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:05.444768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:05.445036 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] PATH_INFO: `/v3/users/05ca5b336aa44b1180b569205a66e1a6` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:05.446173 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=05ca5b336aa44b1180b569205a66e1a6)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:05.449120 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:05.646895 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05ca5b336aa44b1180b569205a66e1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.647555 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05ca5b336aa44b1180b569205a66e1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.648197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05ca5b336aa44b1180b569205a66e1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.656197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05ca5b336aa44b1180b569205a66e1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.658149 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:05.674904 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-eb9089c3-1239-40bc-873c-e364762daec2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05ca5b336aa44b1180b569205a66e1a6'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:05.769508 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1466/2928] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:16:05 2022] DELETE /identity/v3/users/05ca5b336aa44b1180b569205a66e1a6 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:06.192631 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:06.194151 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:06.207807 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dbcd6-9f36-4a60-b600-5a6cf88594ca admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:06.243531 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:06.273546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:06.274189 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:06.274604 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:06.274884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] PATH_INFO: `/v3/projects/b056acb21b0b4f818327ca72dbd31684` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:06.276100 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=b056acb21b0b4f818327ca72dbd31684)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:06.278106 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:06.513472 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b056acb21b0b4f818327ca72dbd31684', 'request_id': 'req-94bc3674-9c41-4624-9593-a4fc1771e5ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:06.519442 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b056acb21b0b4f818327ca72dbd31684', 'request_id': 'req-94bc3674-9c41-4624-9593-a4fc1771e5ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:06.525407 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b056acb21b0b4f818327ca72dbd31684', 'request_id': 'req-94bc3674-9c41-4624-9593-a4fc1771e5ab'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:06.626426 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] The token cache is being invalidate because project b056acb21b0b4f818327ca72dbd31684 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:16:06.626718 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:06.627347 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-94bc3674-9c41-4624-9593-a4fc1771e5ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:06.628842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1463/2929] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:16:06 2022] DELETE /identity/v3/projects/b056acb21b0b4f818327ca72dbd31684 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:06.633968 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:06.634476 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:06.634583 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:06.656016 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:06.657234 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:06.717416 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1467/2930] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:06.727434 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:06.727714 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:06.727982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:06.748260 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:06.750288 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:06.758940 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1464/2931] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:06.763468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:06.773289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267778e9-f100-41eb-8794-f1c88f77beb5 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:06.777867 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:06.778468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:06.778753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:06.779028 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:06.779674 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81612) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 23 23:16:06.779980 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:06.781762 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:06.793469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1468/2932] 10.222.0.73 () {58 vars in 1153 bytes} [Tue Aug 23 23:16:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 23 23:16:06.798656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:06.807855 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695466cf-7915-4190-92bf-305b3dea5587 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:06.813173 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:06.813761 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:06.814033 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:06.814298 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] PATH_INFO: `/v3/projects` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:06.814923 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:06.816790 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:06.953140 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1465/2933] 10.222.0.73 () {60 vars in 1153 bytes} [Tue Aug 23 23:16:06 2022] POST /identity/v3/projects => generated 375 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:16:06.957179 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:06.967473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe1b352-a773-4e0a-b76e-b25af735f0a4 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:07.029296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:07.029296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:07.029296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:07.029296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:07.029790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:07.029790 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:07.141789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1469/2934] 10.222.0.73 () {60 vars in 1144 bytes} [Tue Aug 23 23:16:06 2022] POST /identity/v3/users => generated 288 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 23 23:16:07.146030 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:07.157281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7700c8df-3f46-4cbe-9dcc-fe2df2957d32 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:07.163044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:07.163044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:07.163044 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:07.163445 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] PATH_INFO: `/v3/roles` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:07.164080 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:07.165728 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:07.179705 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1466/2935] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:16:07.183900 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:07.194473 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1f034c-7fe2-44b0-a3be-0947173f49f2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:07.199010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a540612-9d24-46f6-a2c5-26c1ae782830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:07.199613 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a540612-9d24-46f6-a2c5-26c1ae782830 admin admin] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:07.199882 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a540612-9d24-46f6-a2c5-26c1ae782830 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:07.200152 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a540612-9d24-46f6-a2c5-26c1ae782830 admin admin] PATH_INFO: `/v3/roles` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:07.200789 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a540612-9d24-46f6-a2c5-26c1ae782830 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:07.202601 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a540612-9d24-46f6-a2c5-26c1ae782830 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:07.215045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1470/2936] 10.222.0.73 () {58 vars in 1122 bytes} [Tue Aug 23 23:16:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 23 23:16:07.220145 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:07.230679 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa7662f-88ac-4763-b06b-42ab9032d346 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:07.236902 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:07.237618 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] REQUEST_METHOD: `PUT` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:07.237903 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:07.238176 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] PATH_INFO: `/v3/projects/bfbf18ff78ce46d88c1aff7a7651daf1/users/2e9f9b299d0f436eb499594099091f01/roles/87ab2740a8bb4a509daf398f95e522b1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:07.253774 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfbf18ff78ce46d88c1aff7a7651daf1, user_id=2e9f9b299d0f436eb499594099091f01, role_id=87ab2740a8bb4a509daf398f95e522b1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:07.256658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:07.335823 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1467/2937] 10.222.0.73 () {60 vars in 1483 bytes} [Tue Aug 23 23:16:07 2022] PUT /identity/v3/projects/bfbf18ff78ce46d88c1aff7a7651daf1/users/2e9f9b299d0f436eb499594099091f01/roles/87ab2740a8bb4a509daf398f95e522b1 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:07.347512 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:07.347827 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:07.348107 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:07.376891 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] MFA Rules not processed for user `2e9f9b299d0f436eb499594099091f01`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:07.378119 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:07.408264 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:07.442652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1471/2938] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:07 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:07.454734 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:07.456552 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:07.516025 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2469111b-686f-4e61-8c19-5c908ee54cdc admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:07.519783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c498027e07ce4bc28bc0518f22047b16', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:07.520391 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:07.520678 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:07.520947 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:07.529265 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:07.530732 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:07.536376 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:07.541264 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1468/2939] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:07 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:14.679535 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:14.680965 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:14.750916 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ac1e6b-e1c2-49ce-8cf7-031a994bb146 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:14.754678 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:14.755304 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:14.755585 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:14.755855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:14.758064 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:14.826195 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:14.828711 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:14.839022 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1472/2940] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:21.639043 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:21.640987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:21.681768 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:21.691387 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa380926-28ae-4245-bfea-aeb0a077c25a service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:21.695431 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:21.696075 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:21.696360 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:21.696633 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:21.698433 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:21.703845 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987d2fe-f1e0-469e-bd30-682b677b4afd service glance] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:21.708204 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f17bae22fe2b4aa09619bdc0b3e0b4a3', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:21.708839 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:21.709159 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:21.709432 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:21.711239 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:21.764565 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:21.767526 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:21.771598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:21.773051 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:21.777947 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1473/2941] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 23 23:16:21.782394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1469/2942] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:29.446056 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:29.448518 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:29.462282 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf0541-6225-4c93-801b-bf5e95341dd9 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:29.468785 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:29.469834 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:29.470325 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:29.470966 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] PATH_INFO: `/v3/users/daf49a1b3de0478a939694287535607f` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:29.472769 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daf49a1b3de0478a939694287535607f)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:29.476314 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:29.509807 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:29.731093 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daf49a1b3de0478a939694287535607f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:29.731822 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daf49a1b3de0478a939694287535607f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:29.732469 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daf49a1b3de0478a939694287535607f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:29.740249 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daf49a1b3de0478a939694287535607f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:29.752977 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daf49a1b3de0478a939694287535607f'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:29.823474 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1474/2943] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:16:29 2022] DELETE /identity/v3/users/daf49a1b3de0478a939694287535607f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:30.091540 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:30.109102 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3eeee-0e93-4661-a09e-be27a81e6e8b service cinder] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:30.164149 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:30.164783 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:30.165092 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:30.165370 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] PATH_INFO: `/v3/projects/aa51baae7e1f4d318a3c9607d5cb715d` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:30.166545 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa51baae7e1f4d318a3c9607d5cb715d)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:30.168440 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:30.330891 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa51baae7e1f4d318a3c9607d5cb715d', 'request_id': 'req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.335511 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:30.336638 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa51baae7e1f4d318a3c9607d5cb715d', 'request_id': 'req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.342900 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa51baae7e1f4d318a3c9607d5cb715d', 'request_id': 'req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.489296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] The token cache is being invalidate because project aa51baae7e1f4d318a3c9607d5cb715d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:16:30.489683 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.490337 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.491829 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1470/2944] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:16:30 2022] DELETE /identity/v3/projects/aa51baae7e1f4d318a3c9607d5cb715d => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:30.495788 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:30.497226 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:30.511065 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003bebed-02ab-473a-ab82-62af91c21556 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:30.571614 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:30.574985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:30.574985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:30.574985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] PATH_INFO: `/v3/users/170b0fca9092474fa004b5209d2d96f1` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:30.574985 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170b0fca9092474fa004b5209d2d96f1)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:30.576088 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:30.897970 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170b0fca9092474fa004b5209d2d96f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.898536 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170b0fca9092474fa004b5209d2d96f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.899144 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170b0fca9092474fa004b5209d2d96f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.906123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170b0fca9092474fa004b5209d2d96f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:30.918798 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-22346b8e-b6f1-4ac2-a53b-0350a0891704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170b0fca9092474fa004b5209d2d96f1'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:31.009912 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1475/2945] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:16:30 2022] DELETE /identity/v3/users/170b0fca9092474fa004b5209d2d96f1 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:31.307231 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:31.308746 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:31.322720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9aa320f-b43c-4cbd-988d-2dd28c0c145f admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:31.381628 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:31.382285 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:31.382589 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:31.383115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] PATH_INFO: `/v3/projects/30a63975c48646f4ad8f98da2256028c` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:31.384348 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30a63975c48646f4ad8f98da2256028c)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:31.386256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:31.507154 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30a63975c48646f4ad8f98da2256028c', 'request_id': 'req-42952b1b-b74e-414b-ba1f-48c9f1587000'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:31.512905 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30a63975c48646f4ad8f98da2256028c', 'request_id': 'req-42952b1b-b74e-414b-ba1f-48c9f1587000'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:31.519675 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30a63975c48646f4ad8f98da2256028c', 'request_id': 'req-42952b1b-b74e-414b-ba1f-48c9f1587000'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:31.650356 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] The token cache is being invalidate because project 30a63975c48646f4ad8f98da2256028c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:16:31.651605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:31.651605 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:31.656975 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1471/2946] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:16:31 2022] DELETE /identity/v3/projects/30a63975c48646f4ad8f98da2256028c => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:41.006289 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=74,keystone:DELETE=19,keystone:INSERT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:41.646656 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=56,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:43.860343 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] REQUEST_METHOD: `POST` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:43.860888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:43.860888 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:43.881766 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.auth.core [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] MFA Rules not processed for user `51af1e82755e449993e110db493b689b`. Rule list: `[]` (Enabled: `True`). {{(pid=81612) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:43.882961 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:43.939763 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1476/2947] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:43 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 23 23:16:43.963247 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:43.964720 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:44.031473 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42952b1b-b74e-414b-ba1f-48c9f1587000 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:44.037400 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:44.038463 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:44.038749 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:44.039022 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] PATH_INFO: `/v3/users/2e9f9b299d0f436eb499594099091f01` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:44.040175 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e9f9b299d0f436eb499594099091f01)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:44.042658 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:44.222598 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e9f9b299d0f436eb499594099091f01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:44.223476 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e9f9b299d0f436eb499594099091f01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:44.237747 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e9f9b299d0f436eb499594099091f01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:44.244296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e9f9b299d0f436eb499594099091f01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:44.245875 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-99a1d0f9-1506-4fe5-9a91-809bebf62bae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e9f9b299d0f436eb499594099091f01'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:44.277590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:44.279051 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:44.338167 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7757793-8907-424c-8e93-8556cd70ce56 None None] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:44.341842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:44.342448 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:44.342731 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:44.343006 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:44.344758 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:44.356270 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:44.357984 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:44.358955 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1472/2948] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:16:43 2022] DELETE /identity/v3/users/2e9f9b299d0f436eb499594099091f01 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:44.363256 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] REQUEST_METHOD: `POST` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:44.369546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:44.369546 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:44.369767 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1477/2949] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:44 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:44.384612 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.auth.core [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] MFA Rules not processed for user `0f6d02bd20804957aa92efbfa08393a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81613) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 23 23:16:44.385983 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:44.453977 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1473/2950] 10.222.0.73 () {58 vars in 958 bytes} [Tue Aug 23 23:16:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 23 23:16:44.463829 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:44.465544 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:44.476324 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef32921-5729-4571-8443-aad5de51b4f6 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:44.529123 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:44.529730 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:44.530010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:44.530283 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:44.540033 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:44.541547 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:44.556253 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1478/2951] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:16:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:16:45.098185 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:16:45.100587 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:16:45.114453 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88a9be4-21bf-42ed-a0f7-3b087f62ada1 None None] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:16:45.119987 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:16:45.120569 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:16:45.120849 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:16:45.126115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] PATH_INFO: `/v3/projects/bfbf18ff78ce46d88c1aff7a7651daf1` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:16:45.126115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfbf18ff78ce46d88c1aff7a7651daf1)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:16:45.126115 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:16:45.142812 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:45.302984 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfbf18ff78ce46d88c1aff7a7651daf1', 'request_id': 'req-e3d90c94-53bd-4c94-bc90-16a55e2747aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:45.308676 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfbf18ff78ce46d88c1aff7a7651daf1', 'request_id': 'req-e3d90c94-53bd-4c94-bc90-16a55e2747aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:45.315002 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfbf18ff78ce46d88c1aff7a7651daf1', 'request_id': 'req-e3d90c94-53bd-4c94-bc90-16a55e2747aa'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:45.410068 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] The token cache is being invalidate because project bfbf18ff78ce46d88c1aff7a7651daf1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:16:45.410363 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:45.410995 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:16:45.412787 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1474/2952] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:16:45 2022] DELETE /identity/v3/projects/bfbf18ff78ce46d88c1aff7a7651daf1 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:16:54.547137 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=77 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:16:55.718128 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=18,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:17:03.206333 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:17:03.216168 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b1318-baaf-4df7-b37f-e6caec8e2653 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:17:03.268086 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:17:03.268691 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:17:03.268971 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:17:03.269274 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:17:03.271011 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:17:03.334519 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:17:03.336815 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:17:03.347151 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1479/2953] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:17:03 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:17:13.342395 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=50 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:17:34.507436 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:17:34.508893 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:17:34.535311 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d90c94-53bd-4c94-bc90-16a55e2747aa admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:17:34.538980 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:17:34.539580 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:17:34.539862 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:17:34.540197 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:17:34.542021 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:17:34.552525 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:17:34.553982 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:17:34.562653 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1475/2954] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:17:34 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:17:44.558609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=11 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:19:05.244525 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:05.246164 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:05.359313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fc2a4c-f730-4481-bff5-37caa0c334c2 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:05.366014 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '059f72c1793e492fb052bebbf29b5bec', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:05.366644 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:05.366924 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:05.367197 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:05.368992 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:05.382299 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:05.383918 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:05.397263 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1480/2955] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:19:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:19:05.665214 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:05.674719 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21806765-fc6d-432b-8fcc-681efbd1ff98 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:05.679481 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:05.680207 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:05.680550 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:05.680920 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:05.682868 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:05.697928 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:05.699384 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:05.712609 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1476/2956] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:19:05 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:19:06.910468 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:06.912345 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:06.924432 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe12be5-3d42-4b02-88f7-1b5d7facef80 service nova] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:06.928883 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:06.929542 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:06.929821 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:06.930094 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:06.931859 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:06.948055 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:06.950045 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:06.963439 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1481/2957] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:19:06 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:19:15.706078 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=6 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:19:16.957124 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=52 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:19:22.330864 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:22.340227 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdfaa60-a3b9-49c6-bc1a-35cbe3e9342b service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:22.344076 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'b05fdc39b370474fa59a63b4b0c7748a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:22.344680 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] REQUEST_METHOD: `GET` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:22.344959 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:22.345267 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:22.347649 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:22.442832 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:22.444170 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:22.464075 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1477/2958] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:19:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:19:30.998221 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:30.999826 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:31.014273 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2d106c-214a-4f7d-aa61-2f34385ef300 service neutron] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:31.019855 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:31.020502 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:31.020783 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:31.021074 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] PATH_INFO: `/v3/users/51af1e82755e449993e110db493b689b` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:31.022386 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51af1e82755e449993e110db493b689b)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:31.024309 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:31.210858 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51af1e82755e449993e110db493b689b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.211590 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51af1e82755e449993e110db493b689b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.212251 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51af1e82755e449993e110db493b689b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.220067 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51af1e82755e449993e110db493b689b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.234696 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51af1e82755e449993e110db493b689b'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.316719 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1482/2959] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:19:30 2022] DELETE /identity/v3/users/51af1e82755e449993e110db493b689b => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 23 23:19:31.574884 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:31.586024 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7648d946-c4cc-46da-b42c-da14456e4314 service neutron] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:31.649084 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:31.649740 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:31.650020 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:31.650296 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] PATH_INFO: `/v3/projects/9e5fdd0ced9b4bb3af81a136afaca202` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:31.651559 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e5fdd0ced9b4bb3af81a136afaca202)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:31.653513 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:31.768704 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e5fdd0ced9b4bb3af81a136afaca202', 'request_id': 'req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.775626 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e5fdd0ced9b4bb3af81a136afaca202', 'request_id': 'req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.782387 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e5fdd0ced9b4bb3af81a136afaca202', 'request_id': 'req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.788281 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:19:31.870383 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] The token cache is being invalidate because project 9e5fdd0ced9b4bb3af81a136afaca202 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:19:31.870685 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.871286 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:31.872781 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1478/2960] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:19:31 2022] DELETE /identity/v3/projects/9e5fdd0ced9b4bb3af81a136afaca202 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:19:31.876842 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:31.878353 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:31.892654 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55afc36-f609-4eba-824c-ef11a62002e3 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:31.966090 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:31.966739 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:31.967032 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:31.967337 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] PATH_INFO: `/v3/users/b5ae8f3a7f4944dfbcd33904bee6a9ce` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:31.968504 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5ae8f3a7f4944dfbcd33904bee6a9ce)` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:31.970471 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:32.164753 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5ae8f3a7f4944dfbcd33904bee6a9ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.165852 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5ae8f3a7f4944dfbcd33904bee6a9ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.166923 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5ae8f3a7f4944dfbcd33904bee6a9ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.177652 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5ae8f3a7f4944dfbcd33904bee6a9ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.199524 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.notifications [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5ae8f3a7f4944dfbcd33904bee6a9ce'} {{(pid=81612) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.202357 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=80,keystone:DELETE=19,keystone:INSERT=1 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:19:32.271313 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1483/2961] 10.222.0.73 () {58 vars in 1224 bytes} [Tue Aug 23 23:19:31 2022] DELETE /identity/v3/users/b5ae8f3a7f4944dfbcd33904bee6a9ce => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:19:32.576414 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Authenticating user token {{(pid=81613) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:19:32.578394 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.fernet_utils [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81613) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:19:32.593510 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65aa0c9-dfa8-4afa-b617-d9cb64a7d800 admin admin] Validating token access rules against request {{(pid=81613) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:19:32.678479 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0f6d02bd20804957aa92efbfa08393a0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '6d681c62c63c41f3b7033ae72341d972', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81613) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:19:32.679130 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:19:32.679410 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] SCRIPT_NAME: `/identity` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:19:32.679682 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] PATH_INFO: `/v3/projects/dd212f561fa147ce9d8e64182550fb2e` {{(pid=81613) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:19:32.680842 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd212f561fa147ce9d8e64182550fb2e)` {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:19:32.682792 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] RBAC: Authorization granted {{(pid=81613) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:19:32.870556 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd212f561fa147ce9d8e64182550fb2e', 'request_id': 'req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.877996 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd212f561fa147ce9d8e64182550fb2e', 'request_id': 'req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.885918 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd212f561fa147ce9d8e64182550fb2e', 'request_id': 'req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d'} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.989419 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] The token cache is being invalidate because project dd212f561fa147ce9d8e64182550fb2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81613) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 23 23:19:32.989709 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.990439 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG keystone.notifications [None req-5799ded4-d0a6-4fb4-a7e4-fb1aaf3bb94d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81613) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 23 23:19:32.992379 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: [pid: 81613|app: 0|req: 1479/2962] 10.222.0.73 () {58 vars in 1233 bytes} [Tue Aug 23 23:19:32 2022] DELETE /identity/v3/projects/dd212f561fa147ce9d8e64182550fb2e => generated 0 bytes in 417 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 23 23:19:42.360140 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=2 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:19:42.985310 dv-cinder-842542-4-iscsi devstack@keystone.service[81613]: DEBUG dbcounter [-] [81613] Writing DB stats keystone:SELECT=53,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81613) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 23 23:20:46.763162 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Authenticating user token {{(pid=81612) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 23 23:20:46.764647 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:20:46.897104 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde52d64-8a07-443e-b0b2-b873215ff7d2 admin admin] Validating token access rules against request {{(pid=81612) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 23 23:20:46.959126 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f077a58c9406414085dba8d67c69e364', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'fed3c0cb3baf419aa351a156ce71854b', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81612) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 23 23:20:46.959736 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] REQUEST_METHOD: `GET` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 23 23:20:46.960020 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] SCRIPT_NAME: `/identity` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 23 23:20:46.960296 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81612) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 23 23:20:46.962254 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.fernet_utils [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81612) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 23 23:20:46.974813 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 23 23:20:46.976520 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781fc483-6e99-4891-9ded-8aa5e5acbc4f service placement] RBAC: Authorization granted {{(pid=81612) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 23 23:20:46.987010 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: [pid: 81612|app: 0|req: 1484/2963] 10.222.0.73 () {60 vars in 1365 bytes} [Tue Aug 23 23:20:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 23 23:20:56.982419 dv-cinder-842542-4-iscsi devstack@keystone.service[81612]: DEBUG dbcounter [-] [81612] Writing DB stats keystone:SELECT=84 {{(pid=81612) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}